[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315935617919-15814-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 22:52:15 dist-test-slave-ncp9 krb5kdc[15822](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 22:52:15 dist-test-slave-ncp9 krb5kdc[15822](info): set up 2 sockets
Aug 09 22:52:15 dist-test-slave-ncp9 krb5kdc[15822](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260809 22:52:17.042043 15814 mini_kdc.cc:121] Time spent starting KDC: real 1.331s user 0.006s sys 0.018s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315935617919-15814-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315935617919-15814-0/krb5kdc/test-user.keytab.
Aug 09 22:52:17 dist-test-slave-ncp9 krb5kdc[15822](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315937, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T22:52:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T22:52:17Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.113.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.113.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.113.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.190.keytab.
Entry for principal kudu/127.15.113.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.190.keytab.
I20260809 22:52:17.221644 15814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskItYPJR/build/asan/bin/kudu
/tmp/dist-test-taskItYPJR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315935617919-15814-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315935617919-15814-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315935617919-15814-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315935617919-15814-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.113.190:43587
--webserver_interface=127.15.113.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.190.keytab
--principal=kudu/127.15.113.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.113.148:35627
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.113.190:43587
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315935617919-15814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315935617919-15814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315935617919-15814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:52:17.552830 15849 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:52:17.553180 15849 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:52:17.553310 15849 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:52:17.560451 15849 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 22:52:17.560544 15849 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:52:17.560612 15849 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:52:17.560686 15849 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 22:52:17.560747 15849 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 22:52:17.570220 15849 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.113.148:35627
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315935617919-15814-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315935617919-15814-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.113.190:43587
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.190.keytab
--principal=kudu/127.15.113.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.113.190:43587
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315935617919-15814-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.113.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315935617919-15814-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:52:17.571435 15849 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:52:17.572711 15849 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:52:17.584609 15855 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:52:17.584657 15857 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:52:17.584571 15859 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:52:17.585129 15849 server_base.cc:1034] running on GCE node
I20260809 22:52:17.585872 15849 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:52:17.587185 15849 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:52:17.588385 15849 hybrid_clock.cc:648] HybridClock initialized: now 1786315937588357 us; error 58 us; skew 500 ppm
I20260809 22:52:17.588704 15849 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:52:17 dist-test-slave-ncp9 krb5kdc[15822](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315937, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.113.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:52:17.594244 15849 init.cc:377] Logged in from keytab as kudu/127.15.113.190@KRBTEST.COM (short username kudu)
I20260809 22:52:17.595767 15849 webserver.cc:460] Webserver started at http://127.15.113.190:34177/ using document root <none> and password file <none>
I20260809 22:52:17.596410 15849 fs_manager.cc:359] Metadata directory not provided
I20260809 22:52:17.596529 15849 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:52:17.596827 15849 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:52:17.599982 15849 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315935617919-15814-0/minicluster-data/master-0/data/instance:
uuid: "44f54bc57f05498a88b9a50450fe3f92"
format_stamp: "Formatted at 2026-08-09 22:52:17 on dist-test-slave-ncp9"
server_key: "725a780405f6d5c8f962d6d33bef381a"
server_key_iv: "19361f3b8c5df672fb5d1ef1b89ced73"
server_key_version: "encryptionkey@0"
I20260809 22:52:17.600775 15849 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315935617919-15814-0/minicluster-data/master-0/wal/instance:
uuid: "44f54bc57f05498a88b9a50450fe3f92"
format_stamp: "Formatted at 2026-08-09 22:52:17 on dist-test-slave-ncp9"
server_key: "725a780405f6d5c8f962d6d33bef381a"
server_key_iv: "19361f3b8c5df672fb5d1ef1b89ced73"
server_key_version: "encryptionkey@0"
I20260809 22:52:17.605859 15849 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260809 22:52:17.609344 15870 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:52:17.610558 15849 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 22:52:17.610723 15849 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315935617919-15814-0/minicluster-data/master-0/data,/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315935617919-15814-0/minicluster-data/master-0/wal
uuid: "44f54bc57f05498a88b9a50450fe3f92"
format_stamp: "Formatted at 2026-08-09 22:52:17 on dist-test-slave-ncp9"
server_key: "725a780405f6d5c8f962d6d33bef381a"
server_key_iv: "19361f3b8c5df672fb5d1ef1b89ced73"
server_key_version: "encryptionkey@0"
I20260809 22:52:17.610898 15849 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315935617919-15814-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315935617919-15814-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315935617919-15814-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:52:17.625463 15849 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:52:17.629350 15849 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:52:17.629601 15849 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:52:17.649108 15849 rpc_server.cc:307] RPC server started. Bound to: 127.15.113.190:43587
I20260809 22:52:17.649116 15951 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.113.190:43587 every 8 connection(s)
I20260809 22:52:17.650457 15849 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315935617919-15814-0/minicluster-data/master-0/data/info.pb
I20260809 22:52:17.654318 15952 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:52:17.657094 15814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskItYPJR/build/asan/bin/kudu as pid 15849
I20260809 22:52:17.657433 15814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315935617919-15814-0/minicluster-data/master-0/wal/instance
I20260809 22:52:17.659915 15814 external_mini_cluster.cc:1428] Setting key 5870522e2fdcffe2d348fcf911c51230
I20260809 22:52:17.667364 15952 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 44f54bc57f05498a88b9a50450fe3f92: Bootstrap starting.
I20260809 22:52:17.670320 15952 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 44f54bc57f05498a88b9a50450fe3f92: Neither blocks nor log segments found. Creating new log.
I20260809 22:52:17.671346 15952 log.cc:824] T 00000000000000000000000000000000 P 44f54bc57f05498a88b9a50450fe3f92: Log is configured to *not* fsync() on all Append() calls
I20260809 22:52:17.674059 15952 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 44f54bc57f05498a88b9a50450fe3f92: No bootstrap required, opened a new log
I20260809 22:52:17.683791 15952 raft_consensus.cc:348] T 00000000000000000000000000000000 P 44f54bc57f05498a88b9a50450fe3f92 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44f54bc57f05498a88b9a50450fe3f92" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 43587 } }
I20260809 22:52:17.684312 15952 raft_consensus.cc:374] T 00000000000000000000000000000000 P 44f54bc57f05498a88b9a50450fe3f92 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:52:17.684437 15952 raft_consensus.cc:729] T 00000000000000000000000000000000 P 44f54bc57f05498a88b9a50450fe3f92 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 44f54bc57f05498a88b9a50450fe3f92, State: Initialized, Role: FOLLOWER
I20260809 22:52:17.685257 15952 consensus_queue.cc:260] T 00000000000000000000000000000000 P 44f54bc57f05498a88b9a50450fe3f92 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44f54bc57f05498a88b9a50450fe3f92" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 43587 } }
I20260809 22:52:17.685484 15952 raft_consensus.cc:388] T 00000000000000000000000000000000 P 44f54bc57f05498a88b9a50450fe3f92 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 22:52:17.685606 15952 raft_consensus.cc:482] T 00000000000000000000000000000000 P 44f54bc57f05498a88b9a50450fe3f92 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 22:52:17.685760 15952 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 44f54bc57f05498a88b9a50450fe3f92 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:52:17.688055 15952 raft_consensus.cc:504] T 00000000000000000000000000000000 P 44f54bc57f05498a88b9a50450fe3f92 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44f54bc57f05498a88b9a50450fe3f92" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 43587 } }
I20260809 22:52:17.688551 15952 leader_election.cc:302] T 00000000000000000000000000000000 P 44f54bc57f05498a88b9a50450fe3f92 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 44f54bc57f05498a88b9a50450fe3f92; no voters:
I20260809 22:52:17.689177 15952 leader_election.cc:288] T 00000000000000000000000000000000 P 44f54bc57f05498a88b9a50450fe3f92 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 22:52:17.689265 15961 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 44f54bc57f05498a88b9a50450fe3f92 [term 1 FOLLOWER]: Leader election won for term 1
Aug 09 22:52:17 dist-test-slave-ncp9 krb5kdc[15822](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315937, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.113.190@KRBTEST.COM
I20260809 22:52:17.691267 15961 raft_consensus.cc:686] T 00000000000000000000000000000000 P 44f54bc57f05498a88b9a50450fe3f92 [term 1 LEADER]: Becoming Leader. State: Replica: 44f54bc57f05498a88b9a50450fe3f92, State: Running, Role: LEADER
I20260809 22:52:17.691566 15952 sys_catalog.cc:564] T 00000000000000000000000000000000 P 44f54bc57f05498a88b9a50450fe3f92 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 22:52:17.691944 15961 consensus_queue.cc:237] T 00000000000000000000000000000000 P 44f54bc57f05498a88b9a50450fe3f92 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44f54bc57f05498a88b9a50450fe3f92" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 43587 } }
I20260809 22:52:17.698812 15962 sys_catalog.cc:455] T 00000000000000000000000000000000 P 44f54bc57f05498a88b9a50450fe3f92 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "44f54bc57f05498a88b9a50450fe3f92" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44f54bc57f05498a88b9a50450fe3f92" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 43587 } } }
I20260809 22:52:17.699265 15962 sys_catalog.cc:458] T 00000000000000000000000000000000 P 44f54bc57f05498a88b9a50450fe3f92 [sys.catalog]: This master's current role is: LEADER
I20260809 22:52:17.699802 15964 sys_catalog.cc:455] T 00000000000000000000000000000000 P 44f54bc57f05498a88b9a50450fe3f92 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 44f54bc57f05498a88b9a50450fe3f92. Latest consensus state: current_term: 1 leader_uuid: "44f54bc57f05498a88b9a50450fe3f92" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44f54bc57f05498a88b9a50450fe3f92" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 43587 } } }
I20260809 22:52:17.700067 15964 sys_catalog.cc:458] T 00000000000000000000000000000000 P 44f54bc57f05498a88b9a50450fe3f92 [sys.catalog]: This master's current role is: LEADER
I20260809 22:52:17.703953 15955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:52:17.664812 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53874 (local address 127.15.113.190:43587)
0809 22:52:17.665852 (+ 1040us) server_negotiation.cc:206] Beginning negotiation
0809 22:52:17.665867 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0809 22:52:17.667159 (+ 1292us) server_negotiation.cc:407] Connection header received
0809 22:52:17.676281 (+ 9122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:52:17.676306 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:52:17.676778 (+ 472us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:52:17.677230 (+ 452us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:52:17.679828 (+ 2598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:52:17.681901 (+ 2073us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:52:17.683515 (+ 1614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:52:17.684700 (+ 1185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:52:17.684880 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:52:17.691182 (+ 6302us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:52:17.691195 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:52:17.691217 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:52:17.691274 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:52:17.695883 (+ 4609us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:52:17.697097 (+ 1214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:52:17.697115 (+ 18us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:52:17.697129 (+ 14us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:52:17.697385 (+ 256us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:52:17.698020 (+ 635us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:52:17.698025 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:52:17.698028 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:52:17.698907 (+ 879us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:52:17.699223 (+ 316us) server_negotiation.cc:1035] Waiting for connection context
0809 22:52:17.702037 (+ 2814us) server_negotiation.cc:299] Negotiation successful
0809 22:52:17.702536 (+ 499us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":783,"thread_start_us":488,"threads_started":1}
I20260809 22:52:17.709981 15972 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 22:52:17.715309 15972 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 22:52:17.723867 15972 catalog_manager.cc:1347] Generated new cluster ID: f6c34531444b43a9917f5cac6b7dc4ad
I20260809 22:52:17.724030 15972 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 22:52:17.738379 15972 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 22:52:17.739898 15972 catalog_manager.cc:1504] Loading token signing keys...
I20260809 22:52:17.747866 15972 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 44f54bc57f05498a88b9a50450fe3f92: Generated new TSK 0
I20260809 22:52:17.748605 15972 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.113.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.113.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.113.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.129.keytab.
Entry for principal kudu/127.15.113.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.129.keytab.
I20260809 22:52:17.800499 15814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskItYPJR/build/asan/bin/kudu
/tmp/dist-test-taskItYPJR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315935617919-15814-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315935617919-15814-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315935617919-15814-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315935617919-15814-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.113.129:0
--local_ip_for_outbound_sockets=127.15.113.129
--webserver_interface=127.15.113.129
--webserver_port=0
--tserver_master_addrs=127.15.113.190:43587
--keytab_file=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.129.keytab
--principal=kudu/127.15.113.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.113.148:35627
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315935617919-15814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315935617919-15814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315935617919-15814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:52:18.134699 15989 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:52:18.134974 15989 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:52:18.135113 15989 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:52:18.142199 15989 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:52:18.142309 15989 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:52:18.142453 15989 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.113.129
I20260809 22:52:18.151789 15989 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.113.148:35627
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315935617919-15814-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315935617919-15814-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.129.keytab
--principal=kudu/127.15.113.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.113.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315935617919-15814-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.113.129
--webserver_port=0
--tserver_master_addrs=127.15.113.190:43587
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.113.129
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315935617919-15814-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:52:18.153033 15989 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:52:18.154382 15989 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:52:18.166965 15997 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:52:18.167104 16003 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:52:18.167073 15999 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:52:18.168310 15989 server_base.cc:1034] running on GCE node
I20260809 22:52:18.168825 15989 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:52:18.169723 15989 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:52:18.170917 15989 hybrid_clock.cc:648] HybridClock initialized: now 1786315938170867 us; error 46 us; skew 500 ppm
I20260809 22:52:18.171222 15989 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:52:18 dist-test-slave-ncp9 krb5kdc[15822](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315938, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.113.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:52:18.174441 15989 init.cc:377] Logged in from keytab as kudu/127.15.113.129@KRBTEST.COM (short username kudu)
I20260809 22:52:18.175827 15989 webserver.cc:460] Webserver started at http://127.15.113.129:43463/ using document root <none> and password file <none>
I20260809 22:52:18.176427 15989 fs_manager.cc:359] Metadata directory not provided
I20260809 22:52:18.176525 15989 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:52:18.176797 15989 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:52:18.179668 15989 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315935617919-15814-0/minicluster-data/ts-0/data/instance:
uuid: "e2602e2955a140619fc973e4371a20a2"
format_stamp: "Formatted at 2026-08-09 22:52:18 on dist-test-slave-ncp9"
server_key: "ee40a3982579d900c6d4139280569049"
server_key_iv: "73b813f6f45df4dfea6aed95ee957f2e"
server_key_version: "encryptionkey@0"
I20260809 22:52:18.180351 15989 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315935617919-15814-0/minicluster-data/ts-0/wal/instance:
uuid: "e2602e2955a140619fc973e4371a20a2"
format_stamp: "Formatted at 2026-08-09 22:52:18 on dist-test-slave-ncp9"
server_key: "ee40a3982579d900c6d4139280569049"
server_key_iv: "73b813f6f45df4dfea6aed95ee957f2e"
server_key_version: "encryptionkey@0"
I20260809 22:52:18.185256 15989 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.004s
I20260809 22:52:18.188421 16013 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:52:18.189632 15989 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 22:52:18.189775 15989 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315935617919-15814-0/minicluster-data/ts-0/data,/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315935617919-15814-0/minicluster-data/ts-0/wal
uuid: "e2602e2955a140619fc973e4371a20a2"
format_stamp: "Formatted at 2026-08-09 22:52:18 on dist-test-slave-ncp9"
server_key: "ee40a3982579d900c6d4139280569049"
server_key_iv: "73b813f6f45df4dfea6aed95ee957f2e"
server_key_version: "encryptionkey@0"
I20260809 22:52:18.189940 15989 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315935617919-15814-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315935617919-15814-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315935617919-15814-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:52:18.212249 15989 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:52:18.216749 15989 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:52:18.217041 15989 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:52:18.218053 15989 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:52:18.219375 15989 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:52:18.219461 15989 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:52:18.219591 15989 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:52:18.219656 15989 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:52:18.256533 15989 rpc_server.cc:307] RPC server started. Bound to: 127.15.113.129:42739
I20260809 22:52:18.256567 16189 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.113.129:42739 every 8 connection(s)
I20260809 22:52:18.257828 15989 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315935617919-15814-0/minicluster-data/ts-0/data/info.pb
I20260809 22:52:18.268258 15814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskItYPJR/build/asan/bin/kudu as pid 15989
I20260809 22:52:18.268507 15814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315935617919-15814-0/minicluster-data/ts-0/wal/instance
I20260809 22:52:18.269277 15814 external_mini_cluster.cc:1428] Setting key c46a89b20f53f32aecfe39b8aa7cba63
Aug 09 22:52:18 dist-test-slave-ncp9 krb5kdc[15822](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315938, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.113.129@KRBTEST.COM for kudu/127.15.113.190@KRBTEST.COM
I20260809 22:52:18.278497 16198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:52:18.260555 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.113.129:51441 (local address 127.15.113.190:43587)
0809 22:52:18.261085 (+ 530us) server_negotiation.cc:206] Beginning negotiation
0809 22:52:18.261090 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:52:18.262117 (+ 1027us) server_negotiation.cc:407] Connection header received
0809 22:52:18.264136 (+ 2019us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:52:18.264142 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:52:18.264233 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:52:18.264332 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:52:18.265870 (+ 1538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:52:18.266589 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:52:18.268338 (+ 1749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:52:18.268795 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:52:18.268938 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:52:18.272805 (+ 3867us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:52:18.272810 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:52:18.272813 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:52:18.272844 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:52:18.275327 (+ 2483us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:52:18.276199 (+ 872us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:52:18.276203 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:52:18.276205 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:52:18.276326 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:52:18.276808 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:52:18.276811 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:52:18.276813 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:52:18.277111 (+ 298us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:52:18.277229 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0809 22:52:18.278074 (+ 845us) server_negotiation.cc:299] Negotiation successful
0809 22:52:18.278241 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":421,"thread_start_us":266,"threads_started":1}
I20260809 22:52:18.279847 16200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:52:18.261178 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:43587 (local address 127.15.113.129:51441)
0809 22:52:18.261945 (+ 767us) negotiation.cc:107] Waiting for socket to connect
0809 22:52:18.261988 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 22:52:18.263834 (+ 1846us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:52:18.264600 (+ 766us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:52:18.264608 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:52:18.265138 (+ 530us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:52:18.265708 (+ 570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:52:18.265725 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:52:18.266751 (+ 1026us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:52:18.266755 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:52:18.268167 (+ 1412us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:52:18.268184 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:52:18.269064 (+ 880us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:52:18.269073 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:52:18.269363 (+ 290us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:52:18.270047 (+ 684us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:52:18.270069 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:52:18.272603 (+ 2534us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:52:18.275474 (+ 2871us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:52:18.275483 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:52:18.275499 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:52:18.276087 (+ 588us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:52:18.276424 (+ 337us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:52:18.276430 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:52:18.276433 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:52:18.276701 (+ 268us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:52:18.277240 (+ 539us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:52:18.277247 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:52:18.277688 (+ 441us) client_negotiation.cc:769] Sending connection context
0809 22:52:18.277998 (+ 310us) client_negotiation.cc:240] Negotiation successful
0809 22:52:18.278489 (+ 491us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":533,"thread_start_us":329,"threads_started":1}
I20260809 22:52:18.281314 16194 heartbeater.cc:347] Connected to a master server at 127.15.113.190:43587
I20260809 22:52:18.281605 16194 heartbeater.cc:464] Registering TS with master...
I20260809 22:52:18.282478 16194 heartbeater.cc:511] Master 127.15.113.190:43587 requested a full tablet report, sending...
I20260809 22:52:18.284735 15899 ts_manager.cc:194] Registered new tserver with Master: e2602e2955a140619fc973e4371a20a2 (127.15.113.129:42739)
I20260809 22:52:18.286202 15899 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.113.129@KRBTEST.COM'} at 127.15.113.129:51441
WARNING: no policy specified for kudu/127.15.113.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.113.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.113.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.130.keytab.
Entry for principal kudu/127.15.113.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.130.keytab.
I20260809 22:52:18.318102 15814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskItYPJR/build/asan/bin/kudu
/tmp/dist-test-taskItYPJR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315935617919-15814-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315935617919-15814-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315935617919-15814-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315935617919-15814-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.113.130:0
--local_ip_for_outbound_sockets=127.15.113.130
--webserver_interface=127.15.113.130
--webserver_port=0
--tserver_master_addrs=127.15.113.190:43587
--keytab_file=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.130.keytab
--principal=kudu/127.15.113.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.113.148:35627
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315935617919-15814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315935617919-15814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315935617919-15814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:52:18.645227 16203 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:52:18.645485 16203 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:52:18.645619 16203 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:52:18.652635 16203 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:52:18.652729 16203 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:52:18.652869 16203 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.113.130
I20260809 22:52:18.662341 16203 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.113.148:35627
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315935617919-15814-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315935617919-15814-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.130.keytab
--principal=kudu/127.15.113.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.113.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315935617919-15814-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.113.130
--webserver_port=0
--tserver_master_addrs=127.15.113.190:43587
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.113.130
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315935617919-15814-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:52:18.663481 16203 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:52:18.664755 16203 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:52:18.677115 16208 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:52:18.677156 16212 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:52:18.677131 16210 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:52:18.678308 16203 server_base.cc:1034] running on GCE node
I20260809 22:52:18.678867 16203 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:52:18.679711 16203 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:52:18.680935 16203 hybrid_clock.cc:648] HybridClock initialized: now 1786315938680848 us; error 50 us; skew 500 ppm
I20260809 22:52:18.681223 16203 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:52:18 dist-test-slave-ncp9 krb5kdc[15822](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315938, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.113.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:52:18.684144 16203 init.cc:377] Logged in from keytab as kudu/127.15.113.130@KRBTEST.COM (short username kudu)
I20260809 22:52:18.685597 16203 webserver.cc:460] Webserver started at http://127.15.113.130:32865/ using document root <none> and password file <none>
I20260809 22:52:18.686224 16203 fs_manager.cc:359] Metadata directory not provided
I20260809 22:52:18.686340 16203 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:52:18.686659 16203 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:52:18.689615 16203 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315935617919-15814-0/minicluster-data/ts-1/data/instance:
uuid: "70f27a34841e422d920623f503321719"
format_stamp: "Formatted at 2026-08-09 22:52:18 on dist-test-slave-ncp9"
server_key: "bbc640b4d66d55de9e2ba43956c1527a"
server_key_iv: "b194f4ae01c48478bd383d19fcf1fdad"
server_key_version: "encryptionkey@0"
I20260809 22:52:18.690378 16203 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315935617919-15814-0/minicluster-data/ts-1/wal/instance:
uuid: "70f27a34841e422d920623f503321719"
format_stamp: "Formatted at 2026-08-09 22:52:18 on dist-test-slave-ncp9"
server_key: "bbc640b4d66d55de9e2ba43956c1527a"
server_key_iv: "b194f4ae01c48478bd383d19fcf1fdad"
server_key_version: "encryptionkey@0"
I20260809 22:52:18.696051 16203 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260809 22:52:18.699685 16227 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:52:18.700982 16203 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 22:52:18.701135 16203 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315935617919-15814-0/minicluster-data/ts-1/data,/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315935617919-15814-0/minicluster-data/ts-1/wal
uuid: "70f27a34841e422d920623f503321719"
format_stamp: "Formatted at 2026-08-09 22:52:18 on dist-test-slave-ncp9"
server_key: "bbc640b4d66d55de9e2ba43956c1527a"
server_key_iv: "b194f4ae01c48478bd383d19fcf1fdad"
server_key_version: "encryptionkey@0"
I20260809 22:52:18.701313 16203 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315935617919-15814-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315935617919-15814-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315935617919-15814-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:52:18.715232 16203 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:52:18.718972 16203 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:52:18.719240 16203 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:52:18.720147 16203 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:52:18.721513 16203 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:52:18.721591 16203 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:52:18.721721 16203 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:52:18.721779 16203 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:52:18.756512 16203 rpc_server.cc:307] RPC server started. Bound to: 127.15.113.130:37987
I20260809 22:52:18.756542 16399 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.113.130:37987 every 8 connection(s)
I20260809 22:52:18.757891 16203 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315935617919-15814-0/minicluster-data/ts-1/data/info.pb
I20260809 22:52:18.762627 15814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskItYPJR/build/asan/bin/kudu as pid 16203
I20260809 22:52:18.762818 15814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315935617919-15814-0/minicluster-data/ts-1/wal/instance
I20260809 22:52:18.763263 15814 external_mini_cluster.cc:1428] Setting key 91ec6a9efc477ff4b4018e137ceb7850
Aug 09 22:52:18 dist-test-slave-ncp9 krb5kdc[15822](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315938, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.113.130@KRBTEST.COM for kudu/127.15.113.190@KRBTEST.COM
I20260809 22:52:18.777426 16198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:52:18.760448 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.113.130:43281 (local address 127.15.113.190:43587)
0809 22:52:18.760605 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0809 22:52:18.760610 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:52:18.761892 (+ 1282us) server_negotiation.cc:407] Connection header received
0809 22:52:18.763825 (+ 1933us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:52:18.763833 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:52:18.763915 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:52:18.764017 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:52:18.765667 (+ 1650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:52:18.766649 (+ 982us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:52:18.768143 (+ 1494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:52:18.768768 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:52:18.768931 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:52:18.772052 (+ 3121us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:52:18.772059 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:52:18.772064 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:52:18.772102 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:52:18.774601 (+ 2499us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:52:18.775259 (+ 658us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:52:18.775264 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:52:18.775266 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:52:18.775401 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:52:18.775800 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:52:18.775803 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:52:18.775805 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:52:18.776069 (+ 264us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:52:18.776203 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0809 22:52:18.776846 (+ 643us) server_negotiation.cc:299] Negotiation successful
0809 22:52:18.777104 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260809 22:52:18.778301 16404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:52:18.760997 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:43587 (local address 127.15.113.130:43281)
0809 22:52:18.761714 (+ 717us) negotiation.cc:107] Waiting for socket to connect
0809 22:52:18.761761 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0809 22:52:18.763540 (+ 1779us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:52:18.764244 (+ 704us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:52:18.764252 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:52:18.764739 (+ 487us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:52:18.765458 (+ 719us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:52:18.765479 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:52:18.766776 (+ 1297us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:52:18.766779 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:52:18.767995 (+ 1216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:52:18.768008 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:52:18.769011 (+ 1003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:52:18.769017 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:52:18.769218 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:52:18.769819 (+ 601us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:52:18.769840 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:52:18.771838 (+ 1998us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:52:18.774736 (+ 2898us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:52:18.774743 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:52:18.774753 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:52:18.775125 (+ 372us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:52:18.775504 (+ 379us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:52:18.775508 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:52:18.775510 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:52:18.775686 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:52:18.776203 (+ 517us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:52:18.776210 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:52:18.776498 (+ 288us) client_negotiation.cc:769] Sending connection context
0809 22:52:18.776714 (+ 216us) client_negotiation.cc:240] Negotiation successful
0809 22:52:18.777226 (+ 512us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":482,"thread_start_us":271,"threads_started":1}
I20260809 22:52:18.779808 16400 heartbeater.cc:347] Connected to a master server at 127.15.113.190:43587
I20260809 22:52:18.780079 16400 heartbeater.cc:464] Registering TS with master...
I20260809 22:52:18.780860 16400 heartbeater.cc:511] Master 127.15.113.190:43587 requested a full tablet report, sending...
I20260809 22:52:18.782307 15899 ts_manager.cc:194] Registered new tserver with Master: 70f27a34841e422d920623f503321719 (127.15.113.130:37987)
I20260809 22:52:18.783080 15899 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.113.130@KRBTEST.COM'} at 127.15.113.130:43281
WARNING: no policy specified for kudu/127.15.113.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.113.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.113.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.131.keytab.
Entry for principal kudu/127.15.113.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.131.keytab.
I20260809 22:52:18.806262 15814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskItYPJR/build/asan/bin/kudu
/tmp/dist-test-taskItYPJR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315935617919-15814-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315935617919-15814-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315935617919-15814-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315935617919-15814-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.113.131:0
--local_ip_for_outbound_sockets=127.15.113.131
--webserver_interface=127.15.113.131
--webserver_port=0
--tserver_master_addrs=127.15.113.190:43587
--keytab_file=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.131.keytab
--principal=kudu/127.15.113.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.113.148:35627
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315935617919-15814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315935617919-15814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315935617919-15814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:52:19.131492 16408 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:52:19.131768 16408 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:52:19.131907 16408 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:52:19.139000 16408 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:52:19.139101 16408 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:52:19.139267 16408 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.113.131
I20260809 22:52:19.148818 16408 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.113.148:35627
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315935617919-15814-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315935617919-15814-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.131.keytab
--principal=kudu/127.15.113.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.113.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315935617919-15814-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.113.131
--webserver_port=0
--tserver_master_addrs=127.15.113.190:43587
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.113.131
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315935617919-15814-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:52:19.150069 16408 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:52:19.151381 16408 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:52:19.163944 16415 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:52:19.164068 16414 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:52:19.164108 16419 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:52:19.165019 16408 server_base.cc:1034] running on GCE node
I20260809 22:52:19.165562 16408 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:52:19.166402 16408 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:52:19.167596 16408 hybrid_clock.cc:648] HybridClock initialized: now 1786315939167549 us; error 32 us; skew 500 ppm
I20260809 22:52:19.167881 16408 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:52:19 dist-test-slave-ncp9 krb5kdc[15822](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315939, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.113.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:52:19.170889 16408 init.cc:377] Logged in from keytab as kudu/127.15.113.131@KRBTEST.COM (short username kudu)
I20260809 22:52:19.172128 16408 webserver.cc:460] Webserver started at http://127.15.113.131:35847/ using document root <none> and password file <none>
I20260809 22:52:19.172726 16408 fs_manager.cc:359] Metadata directory not provided
I20260809 22:52:19.172821 16408 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:52:19.173118 16408 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:52:19.175869 16408 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315935617919-15814-0/minicluster-data/ts-2/data/instance:
uuid: "caf519329f2f40969b6b72453bf5f787"
format_stamp: "Formatted at 2026-08-09 22:52:19 on dist-test-slave-ncp9"
server_key: "be32e867d4b323d0b09e0929ca391b75"
server_key_iv: "3575a25b0e4705af940bb2a57bddf511"
server_key_version: "encryptionkey@0"
I20260809 22:52:19.176641 16408 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315935617919-15814-0/minicluster-data/ts-2/wal/instance:
uuid: "caf519329f2f40969b6b72453bf5f787"
format_stamp: "Formatted at 2026-08-09 22:52:19 on dist-test-slave-ncp9"
server_key: "be32e867d4b323d0b09e0929ca391b75"
server_key_iv: "3575a25b0e4705af940bb2a57bddf511"
server_key_version: "encryptionkey@0"
I20260809 22:52:19.181559 16408 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.000s
I20260809 22:52:19.184819 16428 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:52:19.186035 16408 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 22:52:19.186187 16408 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315935617919-15814-0/minicluster-data/ts-2/data,/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315935617919-15814-0/minicluster-data/ts-2/wal
uuid: "caf519329f2f40969b6b72453bf5f787"
format_stamp: "Formatted at 2026-08-09 22:52:19 on dist-test-slave-ncp9"
server_key: "be32e867d4b323d0b09e0929ca391b75"
server_key_iv: "3575a25b0e4705af940bb2a57bddf511"
server_key_version: "encryptionkey@0"
I20260809 22:52:19.186365 16408 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315935617919-15814-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315935617919-15814-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315935617919-15814-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:52:19.207561 16408 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:52:19.211568 16408 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:52:19.211818 16408 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:52:19.212754 16408 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:52:19.214039 16408 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:52:19.214114 16408 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:52:19.214277 16408 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:52:19.214329 16408 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:52:19.245550 16408 rpc_server.cc:307] RPC server started. Bound to: 127.15.113.131:40675
I20260809 22:52:19.245592 16599 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.113.131:40675 every 8 connection(s)
I20260809 22:52:19.246886 16408 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315935617919-15814-0/minicluster-data/ts-2/data/info.pb
I20260809 22:52:19.251032 15814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskItYPJR/build/asan/bin/kudu as pid 16408
I20260809 22:52:19.251226 15814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315935617919-15814-0/minicluster-data/ts-2/wal/instance
I20260809 22:52:19.251783 15814 external_mini_cluster.cc:1428] Setting key 9418c24dfe9909fa9ab42303e013315f
Aug 09 22:52:19 dist-test-slave-ncp9 krb5kdc[15822](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315939, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.113.131@KRBTEST.COM for kudu/127.15.113.190@KRBTEST.COM
I20260809 22:52:19.266798 16198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:52:19.249488 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.113.131:36009 (local address 127.15.113.190:43587)
0809 22:52:19.249680 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0809 22:52:19.249685 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:52:19.251061 (+ 1376us) server_negotiation.cc:407] Connection header received
0809 22:52:19.253528 (+ 2467us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:52:19.253536 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:52:19.253636 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:52:19.253729 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:52:19.255510 (+ 1781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:52:19.256363 (+ 853us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:52:19.257932 (+ 1569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:52:19.258329 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:52:19.258435 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:52:19.261678 (+ 3243us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:52:19.261683 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:52:19.261686 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:52:19.261718 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:52:19.264037 (+ 2319us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:52:19.264691 (+ 654us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:52:19.264695 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:52:19.264697 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:52:19.264818 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:52:19.265235 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:52:19.265238 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:52:19.265240 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:52:19.265483 (+ 243us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:52:19.265611 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0809 22:52:19.266245 (+ 634us) server_negotiation.cc:299] Negotiation successful
0809 22:52:19.266463 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260809 22:52:19.267594 16602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:52:19.250018 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:43587 (local address 127.15.113.131:36009)
0809 22:52:19.250853 (+ 835us) negotiation.cc:107] Waiting for socket to connect
0809 22:52:19.250910 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0809 22:52:19.253191 (+ 2281us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:52:19.254026 (+ 835us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:52:19.254038 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:52:19.254689 (+ 651us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:52:19.255327 (+ 638us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:52:19.255354 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:52:19.256511 (+ 1157us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:52:19.256533 (+ 22us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:52:19.257794 (+ 1261us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:52:19.257807 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:52:19.258540 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:52:19.258549 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:52:19.258763 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:52:19.259408 (+ 645us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:52:19.259428 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:52:19.261481 (+ 2053us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:52:19.264167 (+ 2686us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:52:19.264175 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:52:19.264189 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:52:19.264566 (+ 377us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:52:19.264937 (+ 371us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:52:19.264942 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:52:19.264945 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:52:19.265117 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:52:19.265601 (+ 484us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:52:19.265609 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:52:19.265894 (+ 285us) client_negotiation.cc:769] Sending connection context
0809 22:52:19.266109 (+ 215us) client_negotiation.cc:240] Negotiation successful
0809 22:52:19.266563 (+ 454us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":538,"thread_start_us":360,"threads_started":1}
I20260809 22:52:19.268810 16600 heartbeater.cc:347] Connected to a master server at 127.15.113.190:43587
I20260809 22:52:19.269121 16600 heartbeater.cc:464] Registering TS with master...
I20260809 22:52:19.269795 16600 heartbeater.cc:511] Master 127.15.113.190:43587 requested a full tablet report, sending...
I20260809 22:52:19.271071 15899 ts_manager.cc:194] Registered new tserver with Master: caf519329f2f40969b6b72453bf5f787 (127.15.113.131:40675)
I20260809 22:52:19.271843 15899 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.113.131@KRBTEST.COM'} at 127.15.113.131:36009
I20260809 22:52:19.279738 15814 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 22:52:19.289610 16194 heartbeater.cc:503] Master 127.15.113.190:43587 was elected leader, sending a full tablet report...
Aug 09 22:52:19 dist-test-slave-ncp9 krb5kdc[15822](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315939, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 09 22:52:19 dist-test-slave-ncp9 krb5kdc[15822](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315939, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.15.113.129@KRBTEST.COM
I20260809 22:52:19.330163 16612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:52:19.316044 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52066 (local address 127.15.113.129:42739)
0809 22:52:19.316659 (+ 615us) server_negotiation.cc:206] Beginning negotiation
0809 22:52:19.316666 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:52:19.316695 (+ 29us) server_negotiation.cc:407] Connection header received
0809 22:52:19.316792 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:52:19.316798 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:52:19.317080 (+ 282us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:52:19.317214 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:52:19.318514 (+ 1300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:52:19.319487 (+ 973us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:52:19.320645 (+ 1158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:52:19.321229 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:52:19.321322 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:52:19.324836 (+ 3514us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:52:19.324847 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:52:19.324860 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:52:19.324918 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:52:19.327866 (+ 2948us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:52:19.328446 (+ 580us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:52:19.328452 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:52:19.328454 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:52:19.328572 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:52:19.328883 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:52:19.328886 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:52:19.328887 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:52:19.329246 (+ 359us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:52:19.329459 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0809 22:52:19.329687 (+ 228us) server_negotiation.cc:299] Negotiation successful
0809 22:52:19.329872 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":466,"thread_start_us":301,"threads_started":1}
W20260809 22:52:19.331347 16062 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52066
Aug 09 22:52:19 dist-test-slave-ncp9 krb5kdc[15822](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315939, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 09 22:52:19 dist-test-slave-ncp9 krb5kdc[15822](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315939, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.113.129@KRBTEST.COM
I20260809 22:52:19.375200 16612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:52:19.362632 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52072 (local address 127.15.113.129:42739)
0809 22:52:19.362839 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0809 22:52:19.362845 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:52:19.363017 (+ 172us) server_negotiation.cc:407] Connection header received
0809 22:52:19.363202 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:52:19.363207 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:52:19.363298 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:52:19.363382 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:52:19.364643 (+ 1261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:52:19.365379 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:52:19.366567 (+ 1188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:52:19.366991 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:52:19.367095 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:52:19.370689 (+ 3594us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:52:19.370694 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:52:19.370698 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:52:19.370728 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:52:19.372915 (+ 2187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:52:19.373474 (+ 559us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:52:19.373480 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:52:19.373482 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:52:19.373602 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:52:19.373930 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:52:19.373936 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:52:19.373939 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:52:19.374184 (+ 245us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:52:19.374402 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0809 22:52:19.374700 (+ 298us) server_negotiation.cc:299] Negotiation successful
0809 22:52:19.374909 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260809 22:52:19.376552 15814 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskItYPJR/build/asan/bin/kudu pid 15989
I20260809 22:52:19.387522 16612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:52:19.377401 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52088 (local address 127.15.113.129:42739)
0809 22:52:19.377587 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0809 22:52:19.377592 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:52:19.377844 (+ 252us) server_negotiation.cc:407] Connection header received
0809 22:52:19.377959 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:52:19.377962 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:52:19.378013 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:52:19.378077 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:52:19.379096 (+ 1019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:52:19.379731 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:52:19.380862 (+ 1131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:52:19.381267 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:52:19.381352 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:52:19.383329 (+ 1977us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:52:19.383333 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:52:19.383336 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:52:19.383362 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:52:19.385642 (+ 2280us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:52:19.386167 (+ 525us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:52:19.386170 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:52:19.386172 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:52:19.386282 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:52:19.386529 (+ 247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:52:19.386531 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:52:19.386533 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:52:19.386752 (+ 219us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:52:19.386912 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0809 22:52:19.387098 (+ 186us) server_negotiation.cc:299] Negotiation successful
0809 22:52:19.387230 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260809 22:52:19.388134 16184 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:52:19.471726 15814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskItYPJR/build/asan/bin/kudu with pid 15989
I20260809 22:52:19.489269 15814 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskItYPJR/build/asan/bin/kudu pid 16203
Aug 09 22:52:19 dist-test-slave-ncp9 krb5kdc[15822](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315939, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.113.130@KRBTEST.COM
I20260809 22:52:19.504022 16624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:52:19.490117 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40816 (local address 127.15.113.130:37987)
0809 22:52:19.490675 (+ 558us) server_negotiation.cc:206] Beginning negotiation
0809 22:52:19.490682 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:52:19.490707 (+ 25us) server_negotiation.cc:407] Connection header received
0809 22:52:19.490769 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:52:19.490775 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:52:19.490982 (+ 207us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:52:19.491153 (+ 171us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:52:19.492276 (+ 1123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:52:19.493339 (+ 1063us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:52:19.494521 (+ 1182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:52:19.495067 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:52:19.495170 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:52:19.499205 (+ 4035us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:52:19.499217 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:52:19.499230 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:52:19.499261 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:52:19.501816 (+ 2555us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:52:19.502395 (+ 579us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:52:19.502401 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:52:19.502403 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:52:19.502512 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:52:19.502826 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:52:19.502828 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:52:19.502831 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:52:19.503129 (+ 298us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:52:19.503359 (+ 230us) server_negotiation.cc:1035] Waiting for connection context
0809 22:52:19.503557 (+ 198us) server_negotiation.cc:299] Negotiation successful
0809 22:52:19.503739 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":410,"thread_start_us":260,"threads_started":1}
I20260809 22:52:19.505318 16390 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:52:19.584797 15814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskItYPJR/build/asan/bin/kudu with pid 16203
I20260809 22:52:19.604705 15814 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskItYPJR/build/asan/bin/kudu pid 16408
Aug 09 22:52:19 dist-test-slave-ncp9 krb5kdc[15822](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315939, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.113.131@KRBTEST.COM
I20260809 22:52:19.619801 16626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:52:19.605752 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40890 (local address 127.15.113.131:40675)
0809 22:52:19.606338 (+ 586us) server_negotiation.cc:206] Beginning negotiation
0809 22:52:19.606345 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:52:19.606375 (+ 30us) server_negotiation.cc:407] Connection header received
0809 22:52:19.606445 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:52:19.606452 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:52:19.606678 (+ 226us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:52:19.606808 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:52:19.607964 (+ 1156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:52:19.609073 (+ 1109us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:52:19.610262 (+ 1189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:52:19.610822 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:52:19.610915 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:52:19.614780 (+ 3865us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:52:19.614792 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:52:19.614806 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:52:19.614838 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:52:19.617552 (+ 2714us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:52:19.618101 (+ 549us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:52:19.618108 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:52:19.618110 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:52:19.618231 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:52:19.618490 (+ 259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:52:19.618493 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:52:19.618495 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:52:19.618825 (+ 330us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:52:19.619045 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0809 22:52:19.619250 (+ 205us) server_negotiation.cc:299] Negotiation successful
0809 22:52:19.619439 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":439,"thread_start_us":280,"threads_started":1}
I20260809 22:52:19.621089 16594 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:52:19.701987 15814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskItYPJR/build/asan/bin/kudu with pid 16408
I20260809 22:52:19.723742 15814 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskItYPJR/build/asan/bin/kudu pid 15849
I20260809 22:52:19.724859 15945 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:52:19.826133 15814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskItYPJR/build/asan/bin/kudu with pid 15849
2026-08-09T22:52:19Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (4173 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315935617919-15814-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 22:52:19 dist-test-slave-ncp9 krb5kdc[16633](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 22:52:19 dist-test-slave-ncp9 krb5kdc[16633](info): set up 2 sockets
Aug 09 22:52:19 dist-test-slave-ncp9 krb5kdc[16633](info): commencing operation
krb5kdc: starting...
W20260809 22:52:21.005523 15814 mini_kdc.cc:121] Time spent starting KDC: real 1.131s user 0.002s sys 0.018s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315935617919-15814-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315935617919-15814-0/krb5kdc/test-user.keytab.
Aug 09 22:52:21 dist-test-slave-ncp9 krb5kdc[16633](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315941, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T22:52:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T22:52:21Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.113.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.113.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.113.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.190.keytab.
Entry for principal kudu/127.15.113.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.190.keytab.
I20260809 22:52:21.181993 15814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskItYPJR/build/asan/bin/kudu
/tmp/dist-test-taskItYPJR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315935617919-15814-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315935617919-15814-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315935617919-15814-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315935617919-15814-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.113.190:44473
--webserver_interface=127.15.113.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.190.keytab
--principal=kudu/127.15.113.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.113.148:44907
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.113.190:44473
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315935617919-15814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315935617919-15814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315935617919-15814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:52:21.513491 16657 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:52:21.513792 16657 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:52:21.513935 16657 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:52:21.521167 16657 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 22:52:21.521275 16657 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:52:21.521363 16657 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:52:21.521457 16657 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 22:52:21.521521 16657 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 22:52:21.531694 16657 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.113.148:44907
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315935617919-15814-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315935617919-15814-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.113.190:44473
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.190.keytab
--principal=kudu/127.15.113.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.113.190:44473
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315935617919-15814-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.113.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315935617919-15814-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:52:21.533034 16657 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:52:21.534409 16657 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:52:21.546249 16666 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:52:21.546290 16665 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:52:21.546201 16669 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:52:21.547236 16657 server_base.cc:1034] running on GCE node
I20260809 22:52:21.548022 16657 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:52:21.549449 16657 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:52:21.550668 16657 hybrid_clock.cc:648] HybridClock initialized: now 1786315941550617 us; error 32 us; skew 500 ppm
I20260809 22:52:21.551004 16657 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:52:21 dist-test-slave-ncp9 krb5kdc[16633](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315941, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.113.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:52:21.554260 16657 init.cc:377] Logged in from keytab as kudu/127.15.113.190@KRBTEST.COM (short username kudu)
I20260809 22:52:21.555634 16657 webserver.cc:460] Webserver started at http://127.15.113.190:33455/ using document root <none> and password file <none>
I20260809 22:52:21.556286 16657 fs_manager.cc:359] Metadata directory not provided
I20260809 22:52:21.556406 16657 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:52:21.556707 16657 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:52:21.559682 16657 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315935617919-15814-0/minicluster-data/master-0/data/instance:
uuid: "91fb4a39a5534ffa9a29513bde698ec5"
format_stamp: "Formatted at 2026-08-09 22:52:21 on dist-test-slave-ncp9"
server_key: "131f5c973f745ef8e8aa78b6a8733f39"
server_key_iv: "865353d1248a530c2652d924c4fd7f01"
server_key_version: "encryptionkey@0"
I20260809 22:52:21.560421 16657 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315935617919-15814-0/minicluster-data/master-0/wal/instance:
uuid: "91fb4a39a5534ffa9a29513bde698ec5"
format_stamp: "Formatted at 2026-08-09 22:52:21 on dist-test-slave-ncp9"
server_key: "131f5c973f745ef8e8aa78b6a8733f39"
server_key_iv: "865353d1248a530c2652d924c4fd7f01"
server_key_version: "encryptionkey@0"
I20260809 22:52:21.565483 16657 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.003s
I20260809 22:52:21.568857 16678 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:52:21.570190 16657 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 22:52:21.570353 16657 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315935617919-15814-0/minicluster-data/master-0/data,/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315935617919-15814-0/minicluster-data/master-0/wal
uuid: "91fb4a39a5534ffa9a29513bde698ec5"
format_stamp: "Formatted at 2026-08-09 22:52:21 on dist-test-slave-ncp9"
server_key: "131f5c973f745ef8e8aa78b6a8733f39"
server_key_iv: "865353d1248a530c2652d924c4fd7f01"
server_key_version: "encryptionkey@0"
I20260809 22:52:21.570533 16657 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315935617919-15814-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315935617919-15814-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315935617919-15814-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:52:21.583954 16657 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:52:21.587697 16657 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:52:21.587942 16657 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:52:21.604960 16657 rpc_server.cc:307] RPC server started. Bound to: 127.15.113.190:44473
I20260809 22:52:21.604956 16759 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.113.190:44473 every 8 connection(s)
I20260809 22:52:21.606266 16657 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315935617919-15814-0/minicluster-data/master-0/data/info.pb
I20260809 22:52:21.608870 15814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskItYPJR/build/asan/bin/kudu as pid 16657
I20260809 22:52:21.609144 15814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315935617919-15814-0/minicluster-data/master-0/wal/instance
I20260809 22:52:21.609969 15814 external_mini_cluster.cc:1428] Setting key 393576bd155e74d2c280529c82591513
I20260809 22:52:21.610504 16760 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:52:21.624581 16760 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 91fb4a39a5534ffa9a29513bde698ec5: Bootstrap starting.
Aug 09 22:52:21 dist-test-slave-ncp9 krb5kdc[16633](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315941, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.113.190@KRBTEST.COM
I20260809 22:52:21.627743 16760 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 91fb4a39a5534ffa9a29513bde698ec5: Neither blocks nor log segments found. Creating new log.
I20260809 22:52:21.628798 16760 log.cc:824] T 00000000000000000000000000000000 P 91fb4a39a5534ffa9a29513bde698ec5: Log is configured to *not* fsync() on all Append() calls
I20260809 22:52:21.631523 16760 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 91fb4a39a5534ffa9a29513bde698ec5: No bootstrap required, opened a new log
I20260809 22:52:21.633011 16764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:52:21.611971 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60638 (local address 127.15.113.190:44473)
0809 22:52:21.612726 (+ 755us) server_negotiation.cc:206] Beginning negotiation
0809 22:52:21.612737 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 22:52:21.612778 (+ 41us) server_negotiation.cc:407] Connection header received
0809 22:52:21.615028 (+ 2250us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:52:21.615050 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:52:21.615421 (+ 371us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:52:21.615750 (+ 329us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:52:21.617591 (+ 1841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:52:21.618883 (+ 1292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:52:21.620589 (+ 1706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:52:21.621327 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:52:21.621467 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:52:21.626005 (+ 4538us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:52:21.626016 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:52:21.626029 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:52:21.626070 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:52:21.628989 (+ 2919us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:52:21.629633 (+ 644us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:52:21.629643 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:52:21.629649 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:52:21.629825 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:52:21.630178 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:52:21.630183 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:52:21.630186 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:52:21.630716 (+ 530us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:52:21.630985 (+ 269us) server_negotiation.cc:1035] Waiting for connection context
0809 22:52:21.631482 (+ 497us) server_negotiation.cc:299] Negotiation successful
0809 22:52:21.632020 (+ 538us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":556,"thread_start_us":344,"threads_started":1}
I20260809 22:52:21.639902 16760 raft_consensus.cc:348] T 00000000000000000000000000000000 P 91fb4a39a5534ffa9a29513bde698ec5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91fb4a39a5534ffa9a29513bde698ec5" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 44473 } }
I20260809 22:52:21.640338 16760 raft_consensus.cc:374] T 00000000000000000000000000000000 P 91fb4a39a5534ffa9a29513bde698ec5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:52:21.640457 16760 raft_consensus.cc:729] T 00000000000000000000000000000000 P 91fb4a39a5534ffa9a29513bde698ec5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91fb4a39a5534ffa9a29513bde698ec5, State: Initialized, Role: FOLLOWER
I20260809 22:52:21.641059 16760 consensus_queue.cc:260] T 00000000000000000000000000000000 P 91fb4a39a5534ffa9a29513bde698ec5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91fb4a39a5534ffa9a29513bde698ec5" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 44473 } }
I20260809 22:52:21.641276 16760 raft_consensus.cc:388] T 00000000000000000000000000000000 P 91fb4a39a5534ffa9a29513bde698ec5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 22:52:21.641400 16760 raft_consensus.cc:482] T 00000000000000000000000000000000 P 91fb4a39a5534ffa9a29513bde698ec5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 22:52:21.641558 16760 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 91fb4a39a5534ffa9a29513bde698ec5 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:52:21.643590 16760 raft_consensus.cc:504] T 00000000000000000000000000000000 P 91fb4a39a5534ffa9a29513bde698ec5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91fb4a39a5534ffa9a29513bde698ec5" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 44473 } }
I20260809 22:52:21.644057 16760 leader_election.cc:302] T 00000000000000000000000000000000 P 91fb4a39a5534ffa9a29513bde698ec5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 91fb4a39a5534ffa9a29513bde698ec5; no voters:
I20260809 22:52:21.644593 16760 leader_election.cc:288] T 00000000000000000000000000000000 P 91fb4a39a5534ffa9a29513bde698ec5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 22:52:21.644685 16767 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 91fb4a39a5534ffa9a29513bde698ec5 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 22:52:21.645349 16767 raft_consensus.cc:686] T 00000000000000000000000000000000 P 91fb4a39a5534ffa9a29513bde698ec5 [term 1 LEADER]: Becoming Leader. State: Replica: 91fb4a39a5534ffa9a29513bde698ec5, State: Running, Role: LEADER
I20260809 22:52:21.645876 16767 consensus_queue.cc:237] T 00000000000000000000000000000000 P 91fb4a39a5534ffa9a29513bde698ec5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91fb4a39a5534ffa9a29513bde698ec5" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 44473 } }
I20260809 22:52:21.646520 16760 sys_catalog.cc:564] T 00000000000000000000000000000000 P 91fb4a39a5534ffa9a29513bde698ec5 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 22:52:21.648797 16770 sys_catalog.cc:455] T 00000000000000000000000000000000 P 91fb4a39a5534ffa9a29513bde698ec5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "91fb4a39a5534ffa9a29513bde698ec5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91fb4a39a5534ffa9a29513bde698ec5" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 44473 } } }
I20260809 22:52:21.648859 16771 sys_catalog.cc:455] T 00000000000000000000000000000000 P 91fb4a39a5534ffa9a29513bde698ec5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 91fb4a39a5534ffa9a29513bde698ec5. Latest consensus state: current_term: 1 leader_uuid: "91fb4a39a5534ffa9a29513bde698ec5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91fb4a39a5534ffa9a29513bde698ec5" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 44473 } } }
I20260809 22:52:21.649196 16770 sys_catalog.cc:458] T 00000000000000000000000000000000 P 91fb4a39a5534ffa9a29513bde698ec5 [sys.catalog]: This master's current role is: LEADER
I20260809 22:52:21.649227 16771 sys_catalog.cc:458] T 00000000000000000000000000000000 P 91fb4a39a5534ffa9a29513bde698ec5 [sys.catalog]: This master's current role is: LEADER
I20260809 22:52:21.650803 16778 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 22:52:21.655404 16778 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 22:52:21.665233 16778 catalog_manager.cc:1347] Generated new cluster ID: 5ee94419724b41b0b97e8c19cdf9b054
I20260809 22:52:21.665325 16778 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 22:52:21.678395 16778 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 22:52:21.679566 16778 catalog_manager.cc:1504] Loading token signing keys...
I20260809 22:52:21.690466 16778 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 91fb4a39a5534ffa9a29513bde698ec5: Generated new TSK 0
I20260809 22:52:21.691256 16778 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.113.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.113.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.113.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.129.keytab.
Entry for principal kudu/127.15.113.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.129.keytab.
I20260809 22:52:21.745721 15814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskItYPJR/build/asan/bin/kudu
/tmp/dist-test-taskItYPJR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315935617919-15814-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315935617919-15814-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315935617919-15814-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315935617919-15814-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.113.129:0
--local_ip_for_outbound_sockets=127.15.113.129
--webserver_interface=127.15.113.129
--webserver_port=0
--tserver_master_addrs=127.15.113.190:44473
--keytab_file=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.129.keytab
--principal=kudu/127.15.113.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.113.148:44907
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315935617919-15814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315935617919-15814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315935617919-15814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:52:22.069463 16799 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:52:22.069761 16799 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:52:22.069923 16799 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:52:22.077338 16799 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:52:22.077466 16799 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:52:22.077627 16799 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.113.129
I20260809 22:52:22.087198 16799 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.113.148:44907
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315935617919-15814-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315935617919-15814-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.129.keytab
--principal=kudu/127.15.113.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.113.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315935617919-15814-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.113.129
--webserver_port=0
--tserver_master_addrs=127.15.113.190:44473
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.113.129
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315935617919-15814-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:52:22.088423 16799 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:52:22.089821 16799 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:52:22.103027 16808 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:52:22.103060 16811 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:52:22.103106 16805 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:52:22.103804 16799 server_base.cc:1034] running on GCE node
I20260809 22:52:22.104401 16799 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:52:22.105384 16799 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:52:22.106648 16799 hybrid_clock.cc:648] HybridClock initialized: now 1786315942106588 us; error 30 us; skew 500 ppm
I20260809 22:52:22.106961 16799 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:52:22 dist-test-slave-ncp9 krb5kdc[16633](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315942, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.113.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:52:22.109992 16799 init.cc:377] Logged in from keytab as kudu/127.15.113.129@KRBTEST.COM (short username kudu)
I20260809 22:52:22.111382 16799 webserver.cc:460] Webserver started at http://127.15.113.129:38945/ using document root <none> and password file <none>
I20260809 22:52:22.112037 16799 fs_manager.cc:359] Metadata directory not provided
I20260809 22:52:22.112156 16799 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:52:22.112455 16799 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:52:22.115382 16799 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315935617919-15814-0/minicluster-data/ts-0/data/instance:
uuid: "c2e8db58047a476fa02950aac0f295f7"
format_stamp: "Formatted at 2026-08-09 22:52:22 on dist-test-slave-ncp9"
server_key: "40b753ffb6cec12853ef009c135dcc46"
server_key_iv: "e7d60e2715367b07bf4e12238c34139e"
server_key_version: "encryptionkey@0"
I20260809 22:52:22.116211 16799 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315935617919-15814-0/minicluster-data/ts-0/wal/instance:
uuid: "c2e8db58047a476fa02950aac0f295f7"
format_stamp: "Formatted at 2026-08-09 22:52:22 on dist-test-slave-ncp9"
server_key: "40b753ffb6cec12853ef009c135dcc46"
server_key_iv: "e7d60e2715367b07bf4e12238c34139e"
server_key_version: "encryptionkey@0"
I20260809 22:52:22.121135 16799 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260809 22:52:22.124404 16822 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:52:22.125665 16799 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 22:52:22.125810 16799 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315935617919-15814-0/minicluster-data/ts-0/data,/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315935617919-15814-0/minicluster-data/ts-0/wal
uuid: "c2e8db58047a476fa02950aac0f295f7"
format_stamp: "Formatted at 2026-08-09 22:52:22 on dist-test-slave-ncp9"
server_key: "40b753ffb6cec12853ef009c135dcc46"
server_key_iv: "e7d60e2715367b07bf4e12238c34139e"
server_key_version: "encryptionkey@0"
I20260809 22:52:22.125981 16799 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315935617919-15814-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315935617919-15814-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315935617919-15814-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:52:22.151541 16799 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:52:22.155383 16799 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:52:22.155670 16799 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:52:22.156587 16799 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:52:22.157969 16799 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:52:22.158051 16799 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:52:22.158174 16799 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:52:22.158231 16799 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:52:22.188247 16799 rpc_server.cc:307] RPC server started. Bound to: 127.15.113.129:44319
I20260809 22:52:22.188304 16998 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.113.129:44319 every 8 connection(s)
I20260809 22:52:22.189633 16799 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315935617919-15814-0/minicluster-data/ts-0/data/info.pb
I20260809 22:52:22.193034 15814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskItYPJR/build/asan/bin/kudu as pid 16799
I20260809 22:52:22.193290 15814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315935617919-15814-0/minicluster-data/ts-0/wal/instance
I20260809 22:52:22.194059 15814 external_mini_cluster.cc:1428] Setting key 6a9d79d59ce4eb0279c52ab63977e66c
Aug 09 22:52:22 dist-test-slave-ncp9 krb5kdc[16633](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315942, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.113.129@KRBTEST.COM for kudu/127.15.113.190@KRBTEST.COM
I20260809 22:52:22.209195 17002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:52:22.192170 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.113.129:57855 (local address 127.15.113.190:44473)
0809 22:52:22.192634 (+ 464us) server_negotiation.cc:206] Beginning negotiation
0809 22:52:22.192641 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:52:22.193727 (+ 1086us) server_negotiation.cc:407] Connection header received
0809 22:52:22.195918 (+ 2191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:52:22.195922 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:52:22.195983 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:52:22.196076 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:52:22.197612 (+ 1536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:52:22.198333 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:52:22.199834 (+ 1501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:52:22.200246 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:52:22.200353 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:52:22.203720 (+ 3367us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:52:22.203727 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:52:22.203732 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:52:22.203770 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:52:22.206270 (+ 2500us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:52:22.206963 (+ 693us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:52:22.206967 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:52:22.206969 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:52:22.207087 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:52:22.207502 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:52:22.207506 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:52:22.207507 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:52:22.207770 (+ 263us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:52:22.207916 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0809 22:52:22.208583 (+ 667us) server_negotiation.cc:299] Negotiation successful
0809 22:52:22.208807 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":315,"thread_start_us":204,"threads_started":1}
I20260809 22:52:22.210150 17004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:52:22.192760 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:44473 (local address 127.15.113.129:57855)
0809 22:52:22.193574 (+ 814us) negotiation.cc:107] Waiting for socket to connect
0809 22:52:22.193613 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0809 22:52:22.195630 (+ 2017us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:52:22.196326 (+ 696us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:52:22.196333 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:52:22.196848 (+ 515us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:52:22.197454 (+ 606us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:52:22.197472 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:52:22.198475 (+ 1003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:52:22.198478 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:52:22.199695 (+ 1217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:52:22.199708 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:52:22.200456 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:52:22.200464 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:52:22.200703 (+ 239us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:52:22.201326 (+ 623us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:52:22.201348 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:52:22.203507 (+ 2159us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:52:22.206403 (+ 2896us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:52:22.206412 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:52:22.206428 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:52:22.206841 (+ 413us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:52:22.207197 (+ 356us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:52:22.207202 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:52:22.207205 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:52:22.207387 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:52:22.207905 (+ 518us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:52:22.207913 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:52:22.208221 (+ 308us) client_negotiation.cc:769] Sending connection context
0809 22:52:22.208449 (+ 228us) client_negotiation.cc:240] Negotiation successful
0809 22:52:22.209000 (+ 551us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":554,"thread_start_us":319,"threads_started":1}
I20260809 22:52:22.211829 16999 heartbeater.cc:347] Connected to a master server at 127.15.113.190:44473
I20260809 22:52:22.212114 16999 heartbeater.cc:464] Registering TS with master...
I20260809 22:52:22.212807 16999 heartbeater.cc:511] Master 127.15.113.190:44473 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.15.113.130@KRBTEST.COM; defaulting to no policy
I20260809 22:52:22.214846 16711 ts_manager.cc:194] Registered new tserver with Master: c2e8db58047a476fa02950aac0f295f7 (127.15.113.129:44319)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.113.130@KRBTEST.COM" created.
I20260809 22:52:22.216403 16711 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.113.129@KRBTEST.COM'} at 127.15.113.129:57855
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.113.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.130.keytab.
Entry for principal kudu/127.15.113.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.130.keytab.
I20260809 22:52:22.237201 15814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskItYPJR/build/asan/bin/kudu
/tmp/dist-test-taskItYPJR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315935617919-15814-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315935617919-15814-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315935617919-15814-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315935617919-15814-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.113.130:0
--local_ip_for_outbound_sockets=127.15.113.130
--webserver_interface=127.15.113.130
--webserver_port=0
--tserver_master_addrs=127.15.113.190:44473
--keytab_file=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.130.keytab
--principal=kudu/127.15.113.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.113.148:44907
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315935617919-15814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315935617919-15814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315935617919-15814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:52:22.561192 17008 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:52:22.561451 17008 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:52:22.561553 17008 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:52:22.568405 17008 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:52:22.568500 17008 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:52:22.568617 17008 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.113.130
I20260809 22:52:22.577859 17008 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.113.148:44907
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315935617919-15814-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315935617919-15814-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.130.keytab
--principal=kudu/127.15.113.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.113.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315935617919-15814-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.113.130
--webserver_port=0
--tserver_master_addrs=127.15.113.190:44473
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.113.130
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315935617919-15814-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:52:22.578979 17008 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:52:22.580235 17008 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:52:22.592384 17016 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:52:22.592453 17017 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:52:22.592438 17019 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:52:22.593209 17008 server_base.cc:1034] running on GCE node
I20260809 22:52:22.593696 17008 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:52:22.594471 17008 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:52:22.595645 17008 hybrid_clock.cc:648] HybridClock initialized: now 1786315942595586 us; error 51 us; skew 500 ppm
I20260809 22:52:22.595916 17008 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:52:22 dist-test-slave-ncp9 krb5kdc[16633](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315942, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.113.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:52:22.601265 17008 init.cc:377] Logged in from keytab as kudu/127.15.113.130@KRBTEST.COM (short username kudu)
I20260809 22:52:22.602635 17008 webserver.cc:460] Webserver started at http://127.15.113.130:38787/ using document root <none> and password file <none>
I20260809 22:52:22.603252 17008 fs_manager.cc:359] Metadata directory not provided
I20260809 22:52:22.603367 17008 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:52:22.603652 17008 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:52:22.606420 17008 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315935617919-15814-0/minicluster-data/ts-1/data/instance:
uuid: "75717bd9f1d14b3496b152a93b132293"
format_stamp: "Formatted at 2026-08-09 22:52:22 on dist-test-slave-ncp9"
server_key: "6ac5b2f1fea70e19ce8f74f8c6356c9b"
server_key_iv: "bc5f1380077233e25b17220393787448"
server_key_version: "encryptionkey@0"
I20260809 22:52:22.607141 17008 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315935617919-15814-0/minicluster-data/ts-1/wal/instance:
uuid: "75717bd9f1d14b3496b152a93b132293"
format_stamp: "Formatted at 2026-08-09 22:52:22 on dist-test-slave-ncp9"
server_key: "6ac5b2f1fea70e19ce8f74f8c6356c9b"
server_key_iv: "bc5f1380077233e25b17220393787448"
server_key_version: "encryptionkey@0"
I20260809 22:52:22.611965 17008 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.006s
I20260809 22:52:22.615209 17026 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:52:22.616494 17008 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 22:52:22.616626 17008 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315935617919-15814-0/minicluster-data/ts-1/data,/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315935617919-15814-0/minicluster-data/ts-1/wal
uuid: "75717bd9f1d14b3496b152a93b132293"
format_stamp: "Formatted at 2026-08-09 22:52:22 on dist-test-slave-ncp9"
server_key: "6ac5b2f1fea70e19ce8f74f8c6356c9b"
server_key_iv: "bc5f1380077233e25b17220393787448"
server_key_version: "encryptionkey@0"
I20260809 22:52:22.616798 17008 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315935617919-15814-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315935617919-15814-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315935617919-15814-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:52:22.634788 17008 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:52:22.638962 17008 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:52:22.639263 17008 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:52:22.640290 17008 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:52:22.641618 17008 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:52:22.641697 17008 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:52:22.641817 17008 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:52:22.641870 17008 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:52:22.671700 17008 rpc_server.cc:307] RPC server started. Bound to: 127.15.113.130:38889
I20260809 22:52:22.671741 17196 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.113.130:38889 every 8 connection(s)
I20260809 22:52:22.672971 17008 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315935617919-15814-0/minicluster-data/ts-1/data/info.pb
I20260809 22:52:22.683151 15814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskItYPJR/build/asan/bin/kudu as pid 17008
I20260809 22:52:22.683334 15814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315935617919-15814-0/minicluster-data/ts-1/wal/instance
I20260809 22:52:22.683828 15814 external_mini_cluster.cc:1428] Setting key 40ef98dbd48d2433e4a55ed2ec1f46b1
Aug 09 22:52:22 dist-test-slave-ncp9 krb5kdc[16633](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315942, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.113.130@KRBTEST.COM for kudu/127.15.113.190@KRBTEST.COM
I20260809 22:52:22.691782 17002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:52:22.675526 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.113.130:59955 (local address 127.15.113.190:44473)
0809 22:52:22.675713 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0809 22:52:22.675718 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:52:22.676868 (+ 1150us) server_negotiation.cc:407] Connection header received
0809 22:52:22.678664 (+ 1796us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:52:22.678670 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:52:22.678753 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:52:22.678838 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:52:22.680283 (+ 1445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:52:22.681084 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:52:22.682593 (+ 1509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:52:22.683021 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:52:22.683131 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:52:22.686455 (+ 3324us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:52:22.686460 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:52:22.686463 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:52:22.686494 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:52:22.688845 (+ 2351us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:52:22.689515 (+ 670us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:52:22.689520 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:52:22.689524 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:52:22.689649 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:52:22.690046 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:52:22.690050 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:52:22.690053 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:52:22.690317 (+ 264us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:52:22.690454 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0809 22:52:22.691224 (+ 770us) server_negotiation.cc:299] Negotiation successful
0809 22:52:22.691448 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260809 22:52:22.692672 17200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:52:22.676028 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:44473 (local address 127.15.113.130:59955)
0809 22:52:22.676706 (+ 678us) negotiation.cc:107] Waiting for socket to connect
0809 22:52:22.676751 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0809 22:52:22.678393 (+ 1642us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:52:22.679088 (+ 695us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:52:22.679096 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:52:22.679579 (+ 483us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:52:22.680107 (+ 528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:52:22.680122 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:52:22.681233 (+ 1111us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:52:22.681237 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:52:22.682456 (+ 1219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:52:22.682469 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:52:22.683251 (+ 782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:52:22.683259 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:52:22.683490 (+ 231us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:52:22.684116 (+ 626us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:52:22.684141 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:52:22.686257 (+ 2116us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:52:22.688991 (+ 2734us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:52:22.689000 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:52:22.689015 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:52:22.689404 (+ 389us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:52:22.689753 (+ 349us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:52:22.689759 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:52:22.689763 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:52:22.689942 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:52:22.690438 (+ 496us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:52:22.690446 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:52:22.690865 (+ 419us) client_negotiation.cc:769] Sending connection context
0809 22:52:22.691108 (+ 243us) client_negotiation.cc:240] Negotiation successful
0809 22:52:22.691620 (+ 512us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":457,"thread_start_us":261,"threads_started":1}
I20260809 22:52:22.694099 17198 heartbeater.cc:347] Connected to a master server at 127.15.113.190:44473
I20260809 22:52:22.694368 17198 heartbeater.cc:464] Registering TS with master...
I20260809 22:52:22.695020 17198 heartbeater.cc:511] Master 127.15.113.190:44473 requested a full tablet report, sending...
I20260809 22:52:22.696364 16711 ts_manager.cc:194] Registered new tserver with Master: 75717bd9f1d14b3496b152a93b132293 (127.15.113.130:38889)
I20260809 22:52:22.697151 16711 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.113.130@KRBTEST.COM'} at 127.15.113.130:59955
WARNING: no policy specified for kudu/127.15.113.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.113.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.113.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.131.keytab.
Entry for principal kudu/127.15.113.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.131.keytab.
I20260809 22:52:22.726217 15814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskItYPJR/build/asan/bin/kudu
/tmp/dist-test-taskItYPJR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315935617919-15814-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315935617919-15814-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315935617919-15814-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315935617919-15814-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.113.131:0
--local_ip_for_outbound_sockets=127.15.113.131
--webserver_interface=127.15.113.131
--webserver_port=0
--tserver_master_addrs=127.15.113.190:44473
--keytab_file=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.131.keytab
--principal=kudu/127.15.113.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.113.148:44907
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315935617919-15814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315935617919-15814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315935617919-15814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:52:23.042738 17204 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:52:23.042999 17204 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:52:23.043090 17204 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:52:23.050182 17204 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:52:23.050283 17204 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:52:23.050400 17204 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.113.131
I20260809 22:52:23.059973 17204 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.113.148:44907
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315935617919-15814-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315935617919-15814-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.131.keytab
--principal=kudu/127.15.113.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.113.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315935617919-15814-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.113.131
--webserver_port=0
--tserver_master_addrs=127.15.113.190:44473
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.113.131
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315935617919-15814-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:52:23.061251 17204 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:52:23.062583 17204 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:52:23.074914 17214 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:52:23.074947 17218 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:52:23.074986 17215 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:52:23.075664 17204 server_base.cc:1034] running on GCE node
I20260809 22:52:23.076224 17204 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:52:23.077117 17204 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:52:23.078308 17204 hybrid_clock.cc:648] HybridClock initialized: now 1786315943078269 us; error 33 us; skew 500 ppm
I20260809 22:52:23.078581 17204 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:52:23 dist-test-slave-ncp9 krb5kdc[16633](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315943, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.113.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:52:23.081583 17204 init.cc:377] Logged in from keytab as kudu/127.15.113.131@KRBTEST.COM (short username kudu)
I20260809 22:52:23.082993 17204 webserver.cc:460] Webserver started at http://127.15.113.131:40257/ using document root <none> and password file <none>
I20260809 22:52:23.083640 17204 fs_manager.cc:359] Metadata directory not provided
I20260809 22:52:23.083750 17204 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:52:23.084051 17204 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:52:23.086875 17204 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315935617919-15814-0/minicluster-data/ts-2/data/instance:
uuid: "2e993ca7f86e4c65becb4b7e21ca4454"
format_stamp: "Formatted at 2026-08-09 22:52:23 on dist-test-slave-ncp9"
server_key: "e54238f9b29a0609a4f0279e58a3f1b0"
server_key_iv: "a86f98655768aa6cdf0698012e356467"
server_key_version: "encryptionkey@0"
I20260809 22:52:23.087630 17204 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315935617919-15814-0/minicluster-data/ts-2/wal/instance:
uuid: "2e993ca7f86e4c65becb4b7e21ca4454"
format_stamp: "Formatted at 2026-08-09 22:52:23 on dist-test-slave-ncp9"
server_key: "e54238f9b29a0609a4f0279e58a3f1b0"
server_key_iv: "a86f98655768aa6cdf0698012e356467"
server_key_version: "encryptionkey@0"
I20260809 22:52:23.092592 17204 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.004s
I20260809 22:52:23.095891 17230 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:52:23.097221 17204 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260809 22:52:23.097364 17204 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315935617919-15814-0/minicluster-data/ts-2/data,/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315935617919-15814-0/minicluster-data/ts-2/wal
uuid: "2e993ca7f86e4c65becb4b7e21ca4454"
format_stamp: "Formatted at 2026-08-09 22:52:23 on dist-test-slave-ncp9"
server_key: "e54238f9b29a0609a4f0279e58a3f1b0"
server_key_iv: "a86f98655768aa6cdf0698012e356467"
server_key_version: "encryptionkey@0"
I20260809 22:52:23.097532 17204 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315935617919-15814-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315935617919-15814-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315935617919-15814-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:52:23.114240 17204 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:52:23.118321 17204 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:52:23.118583 17204 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:52:23.119513 17204 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:52:23.120815 17204 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:52:23.120910 17204 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:52:23.121035 17204 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:52:23.121089 17204 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:52:23.152206 17204 rpc_server.cc:307] RPC server started. Bound to: 127.15.113.131:34661
I20260809 22:52:23.152242 17407 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.113.131:34661 every 8 connection(s)
I20260809 22:52:23.153532 17204 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315935617919-15814-0/minicluster-data/ts-2/data/info.pb
I20260809 22:52:23.162873 15814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskItYPJR/build/asan/bin/kudu as pid 17204
I20260809 22:52:23.163050 15814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315935617919-15814-0/minicluster-data/ts-2/wal/instance
I20260809 22:52:23.163455 15814 external_mini_cluster.cc:1428] Setting key cf6812d398b02c238eda0db47289db9a
Aug 09 22:52:23 dist-test-slave-ncp9 krb5kdc[16633](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315943, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.113.131@KRBTEST.COM for kudu/127.15.113.190@KRBTEST.COM
I20260809 22:52:23.173153 17002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:52:23.156045 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.113.131:51251 (local address 127.15.113.190:44473)
0809 22:52:23.156260 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0809 22:52:23.156265 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:52:23.157543 (+ 1278us) server_negotiation.cc:407] Connection header received
0809 22:52:23.159444 (+ 1901us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:52:23.159449 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:52:23.159542 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:52:23.159632 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:52:23.161217 (+ 1585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:52:23.162019 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:52:23.163632 (+ 1613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:52:23.164177 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:52:23.164315 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:52:23.168050 (+ 3735us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:52:23.168055 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:52:23.168058 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:52:23.168091 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:52:23.170640 (+ 2549us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:52:23.171286 (+ 646us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:52:23.171289 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:52:23.171291 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:52:23.171403 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:52:23.171726 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:52:23.171729 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:52:23.171731 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:52:23.171986 (+ 255us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:52:23.172091 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0809 22:52:23.172682 (+ 591us) server_negotiation.cc:299] Negotiation successful
0809 22:52:23.172854 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260809 22:52:23.174121 17410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:52:23.156544 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:44473 (local address 127.15.113.131:51251)
0809 22:52:23.157338 (+ 794us) negotiation.cc:107] Waiting for socket to connect
0809 22:52:23.157382 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 22:52:23.159182 (+ 1800us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:52:23.159883 (+ 701us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:52:23.159891 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:52:23.160388 (+ 497us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:52:23.161024 (+ 636us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:52:23.161045 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:52:23.162172 (+ 1127us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:52:23.162176 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:52:23.163483 (+ 1307us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:52:23.163499 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:52:23.164395 (+ 896us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:52:23.164400 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:52:23.164698 (+ 298us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:52:23.165455 (+ 757us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:52:23.165485 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:52:23.167866 (+ 2381us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:52:23.170763 (+ 2897us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:52:23.170770 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:52:23.170781 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:52:23.171194 (+ 413us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:52:23.171474 (+ 280us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:52:23.171478 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:52:23.171480 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:52:23.171647 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:52:23.172075 (+ 428us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:52:23.172081 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:52:23.172367 (+ 286us) client_negotiation.cc:769] Sending connection context
0809 22:52:23.172622 (+ 255us) client_negotiation.cc:240] Negotiation successful
0809 22:52:23.173048 (+ 426us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":522,"thread_start_us":315,"threads_started":1}
I20260809 22:52:23.175419 17408 heartbeater.cc:347] Connected to a master server at 127.15.113.190:44473
I20260809 22:52:23.175700 17408 heartbeater.cc:464] Registering TS with master...
I20260809 22:52:23.176393 17408 heartbeater.cc:511] Master 127.15.113.190:44473 requested a full tablet report, sending...
I20260809 22:52:23.177857 16711 ts_manager.cc:194] Registered new tserver with Master: 2e993ca7f86e4c65becb4b7e21ca4454 (127.15.113.131:34661)
I20260809 22:52:23.178864 16711 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.113.131@KRBTEST.COM'} at 127.15.113.131:51251
I20260809 22:52:23.178896 15814 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 22:52:23.196954 17002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:52:23.184840 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60650 (local address 127.15.113.190:44473)
0809 22:52:23.184995 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0809 22:52:23.184999 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 22:52:23.185456 (+ 457us) server_negotiation.cc:407] Connection header received
0809 22:52:23.185605 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:52:23.185610 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:52:23.185673 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:52:23.185749 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:52:23.187210 (+ 1461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:52:23.188068 (+ 858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:52:23.189383 (+ 1315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:52:23.189775 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:52:23.189881 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:52:23.192072 (+ 2191us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:52:23.192077 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:52:23.192080 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:52:23.192117 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:52:23.194508 (+ 2391us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:52:23.195099 (+ 591us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:52:23.195105 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:52:23.195109 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:52:23.195240 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:52:23.195589 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:52:23.195594 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:52:23.195596 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:52:23.195893 (+ 297us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:52:23.196138 (+ 245us) server_negotiation.cc:1035] Waiting for connection context
0809 22:52:23.196430 (+ 292us) server_negotiation.cc:299] Negotiation successful
0809 22:52:23.196631 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260809 22:52:23.202698 16711 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60650:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 22:52:23.209599 16711 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 22:52:23.219214 16999 heartbeater.cc:503] Master 127.15.113.190:44473 was elected leader, sending a full tablet report...
I20260809 22:52:23.231894 17421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:52:23.221665 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.130:38889 (local address 127.0.0.1:42994)
0809 22:52:23.222356 (+ 691us) negotiation.cc:107] Waiting for socket to connect
0809 22:52:23.222406 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0809 22:52:23.222563 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:52:23.223745 (+ 1182us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:52:23.223753 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:52:23.223785 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:52:23.224174 (+ 389us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:52:23.224186 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:52:23.226336 (+ 2150us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:52:23.226342 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:52:23.228797 (+ 2455us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:52:23.228815 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:52:23.230905 (+ 2090us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:52:23.230912 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:52:23.231113 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:52:23.231169 (+ 56us) client_negotiation.cc:769] Sending connection context
0809 22:52:23.231276 (+ 107us) client_negotiation.cc:240] Negotiation successful
0809 22:52:23.231467 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":451,"thread_start_us":296,"threads_started":1}
I20260809 22:52:23.233516 17422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:52:23.221721 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42994 (local address 127.15.113.130:38889)
0809 22:52:23.223089 (+ 1368us) server_negotiation.cc:206] Beginning negotiation
0809 22:52:23.223097 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 22:52:23.223126 (+ 29us) server_negotiation.cc:407] Connection header received
0809 22:52:23.223224 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:52:23.223232 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:52:23.223487 (+ 255us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:52:23.224378 (+ 891us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:52:23.224595 (+ 217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:52:23.226176 (+ 1581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:52:23.229052 (+ 2876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:52:23.230670 (+ 1618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:52:23.232745 (+ 2075us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:52:23.232958 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0809 22:52:23.233041 (+ 83us) server_negotiation.cc:299] Negotiation successful
0809 22:52:23.233213 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1208,"thread_start_us":1209,"threads_started":1}
I20260809 22:52:23.241396 17421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:52:23.222134 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.129:44319 (local address 127.0.0.1:40642)
0809 22:52:23.232434 (+ 10300us) negotiation.cc:107] Waiting for socket to connect
0809 22:52:23.232454 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 22:52:23.232551 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:52:23.235049 (+ 2498us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:52:23.235055 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:52:23.235085 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:52:23.235292 (+ 207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:52:23.235305 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:52:23.237320 (+ 2015us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:52:23.237326 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:52:23.238811 (+ 1485us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:52:23.238832 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:52:23.240557 (+ 1725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:52:23.240563 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:52:23.240762 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:52:23.240794 (+ 32us) client_negotiation.cc:769] Sending connection context
0809 22:52:23.240921 (+ 127us) client_negotiation.cc:240] Negotiation successful
0809 22:52:23.241056 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10200,"mutex_wait_us":3,"thread_start_us":13557,"threads_started":1}
I20260809 22:52:23.241438 17427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:52:23.222828 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40642 (local address 127.15.113.129:44319)
0809 22:52:23.234422 (+ 11594us) server_negotiation.cc:206] Beginning negotiation
0809 22:52:23.234431 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 22:52:23.234460 (+ 29us) server_negotiation.cc:407] Connection header received
0809 22:52:23.234559 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:52:23.234568 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:52:23.234827 (+ 259us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:52:23.235043 (+ 216us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:52:23.235591 (+ 548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:52:23.237169 (+ 1578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:52:23.238975 (+ 1806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:52:23.240370 (+ 1395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:52:23.240573 (+ 203us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:52:23.240756 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0809 22:52:23.240959 (+ 203us) server_negotiation.cc:299] Negotiation successful
0809 22:52:23.241118 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11405,"thread_start_us":11250,"threads_started":1}
I20260809 22:52:23.241077 17100 tablet_service.cc:1467] Processing CreateTablet for tablet 904f38da63744f4a9073a2bb6a648427 (DEFAULT_TABLE table=test-table [id=751dc160f38e4a478e7da95d4cfc8cb8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:52:23.242868 17100 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 904f38da63744f4a9073a2bb6a648427. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:52:23.244303 17424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:52:23.222746 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.131:34661 (local address 127.0.0.1:56826)
0809 22:52:23.235498 (+ 12752us) negotiation.cc:107] Waiting for socket to connect
0809 22:52:23.235515 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 22:52:23.235617 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:52:23.236603 (+ 986us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:52:23.236608 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:52:23.236651 (+ 43us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:52:23.237482 (+ 831us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:52:23.237497 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:52:23.239211 (+ 1714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:52:23.239216 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:52:23.242010 (+ 2794us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:52:23.242025 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:52:23.243561 (+ 1536us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:52:23.243567 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:52:23.243733 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:52:23.243767 (+ 34us) client_negotiation.cc:769] Sending connection context
0809 22:52:23.243853 (+ 86us) client_negotiation.cc:240] Negotiation successful
0809 22:52:23.243972 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12643,"thread_start_us":14273,"threads_started":1}
I20260809 22:52:23.244383 17428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:52:23.223237 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56826 (local address 127.15.113.131:34661)
0809 22:52:23.235727 (+ 12490us) server_negotiation.cc:206] Beginning negotiation
0809 22:52:23.235736 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 22:52:23.235771 (+ 35us) server_negotiation.cc:407] Connection header received
0809 22:52:23.235876 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:52:23.235884 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:52:23.236118 (+ 234us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:52:23.236320 (+ 202us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:52:23.237698 (+ 1378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:52:23.239056 (+ 1358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:52:23.242173 (+ 3117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:52:23.243353 (+ 1180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:52:23.243563 (+ 210us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:52:23.243730 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0809 22:52:23.243939 (+ 209us) server_negotiation.cc:299] Negotiation successful
0809 22:52:23.244090 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12303,"thread_start_us":12157,"threads_started":1}
I20260809 22:52:23.245556 16899 tablet_service.cc:1467] Processing CreateTablet for tablet 904f38da63744f4a9073a2bb6a648427 (DEFAULT_TABLE table=test-table [id=751dc160f38e4a478e7da95d4cfc8cb8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:52:23.246851 16899 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 904f38da63744f4a9073a2bb6a648427. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:52:23.248201 17300 tablet_service.cc:1467] Processing CreateTablet for tablet 904f38da63744f4a9073a2bb6a648427 (DEFAULT_TABLE table=test-table [id=751dc160f38e4a478e7da95d4cfc8cb8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:52:23.249632 17300 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 904f38da63744f4a9073a2bb6a648427. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:52:23.257674 17431 tablet_bootstrap.cc:492] T 904f38da63744f4a9073a2bb6a648427 P 75717bd9f1d14b3496b152a93b132293: Bootstrap starting.
I20260809 22:52:23.259629 17432 tablet_bootstrap.cc:492] T 904f38da63744f4a9073a2bb6a648427 P c2e8db58047a476fa02950aac0f295f7: Bootstrap starting.
I20260809 22:52:23.261332 17431 tablet_bootstrap.cc:654] T 904f38da63744f4a9073a2bb6a648427 P 75717bd9f1d14b3496b152a93b132293: Neither blocks nor log segments found. Creating new log.
I20260809 22:52:23.262341 17431 log.cc:824] T 904f38da63744f4a9073a2bb6a648427 P 75717bd9f1d14b3496b152a93b132293: Log is configured to *not* fsync() on all Append() calls
I20260809 22:52:23.263111 17432 tablet_bootstrap.cc:654] T 904f38da63744f4a9073a2bb6a648427 P c2e8db58047a476fa02950aac0f295f7: Neither blocks nor log segments found. Creating new log.
I20260809 22:52:23.263841 17436 tablet_bootstrap.cc:492] T 904f38da63744f4a9073a2bb6a648427 P 2e993ca7f86e4c65becb4b7e21ca4454: Bootstrap starting.
I20260809 22:52:23.264410 17432 log.cc:824] T 904f38da63744f4a9073a2bb6a648427 P c2e8db58047a476fa02950aac0f295f7: Log is configured to *not* fsync() on all Append() calls
I20260809 22:52:23.265607 17431 tablet_bootstrap.cc:492] T 904f38da63744f4a9073a2bb6a648427 P 75717bd9f1d14b3496b152a93b132293: No bootstrap required, opened a new log
I20260809 22:52:23.265889 17431 ts_tablet_manager.cc:1397] T 904f38da63744f4a9073a2bb6a648427 P 75717bd9f1d14b3496b152a93b132293: Time spent bootstrapping tablet: real 0.008s user 0.004s sys 0.003s
I20260809 22:52:23.268004 17436 tablet_bootstrap.cc:654] T 904f38da63744f4a9073a2bb6a648427 P 2e993ca7f86e4c65becb4b7e21ca4454: Neither blocks nor log segments found. Creating new log.
I20260809 22:52:23.269253 17436 log.cc:824] T 904f38da63744f4a9073a2bb6a648427 P 2e993ca7f86e4c65becb4b7e21ca4454: Log is configured to *not* fsync() on all Append() calls
I20260809 22:52:23.272141 17436 tablet_bootstrap.cc:492] T 904f38da63744f4a9073a2bb6a648427 P 2e993ca7f86e4c65becb4b7e21ca4454: No bootstrap required, opened a new log
I20260809 22:52:23.272356 17432 tablet_bootstrap.cc:492] T 904f38da63744f4a9073a2bb6a648427 P c2e8db58047a476fa02950aac0f295f7: No bootstrap required, opened a new log
I20260809 22:52:23.272364 17436 ts_tablet_manager.cc:1397] T 904f38da63744f4a9073a2bb6a648427 P 2e993ca7f86e4c65becb4b7e21ca4454: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260809 22:52:23.272634 17432 ts_tablet_manager.cc:1397] T 904f38da63744f4a9073a2bb6a648427 P c2e8db58047a476fa02950aac0f295f7: Time spent bootstrapping tablet: real 0.013s user 0.003s sys 0.007s
I20260809 22:52:23.275705 17431 raft_consensus.cc:348] T 904f38da63744f4a9073a2bb6a648427 P 75717bd9f1d14b3496b152a93b132293 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75717bd9f1d14b3496b152a93b132293" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 38889 } } peers { permanent_uuid: "c2e8db58047a476fa02950aac0f295f7" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 44319 } } peers { permanent_uuid: "2e993ca7f86e4c65becb4b7e21ca4454" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 34661 } }
I20260809 22:52:23.276154 17431 raft_consensus.cc:374] T 904f38da63744f4a9073a2bb6a648427 P 75717bd9f1d14b3496b152a93b132293 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:52:23.276257 17431 raft_consensus.cc:729] T 904f38da63744f4a9073a2bb6a648427 P 75717bd9f1d14b3496b152a93b132293 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75717bd9f1d14b3496b152a93b132293, State: Initialized, Role: FOLLOWER
I20260809 22:52:23.276872 17431 consensus_queue.cc:260] T 904f38da63744f4a9073a2bb6a648427 P 75717bd9f1d14b3496b152a93b132293 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75717bd9f1d14b3496b152a93b132293" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 38889 } } peers { permanent_uuid: "c2e8db58047a476fa02950aac0f295f7" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 44319 } } peers { permanent_uuid: "2e993ca7f86e4c65becb4b7e21ca4454" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 34661 } }
I20260809 22:52:23.277989 17198 heartbeater.cc:503] Master 127.15.113.190:44473 was elected leader, sending a full tablet report...
I20260809 22:52:23.278644 17431 ts_tablet_manager.cc:1428] T 904f38da63744f4a9073a2bb6a648427 P 75717bd9f1d14b3496b152a93b132293: Time spent starting tablet: real 0.013s user 0.009s sys 0.004s
I20260809 22:52:23.280727 17436 raft_consensus.cc:348] T 904f38da63744f4a9073a2bb6a648427 P 2e993ca7f86e4c65becb4b7e21ca4454 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75717bd9f1d14b3496b152a93b132293" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 38889 } } peers { permanent_uuid: "c2e8db58047a476fa02950aac0f295f7" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 44319 } } peers { permanent_uuid: "2e993ca7f86e4c65becb4b7e21ca4454" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 34661 } }
I20260809 22:52:23.281175 17436 raft_consensus.cc:374] T 904f38da63744f4a9073a2bb6a648427 P 2e993ca7f86e4c65becb4b7e21ca4454 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:52:23.281287 17436 raft_consensus.cc:729] T 904f38da63744f4a9073a2bb6a648427 P 2e993ca7f86e4c65becb4b7e21ca4454 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2e993ca7f86e4c65becb4b7e21ca4454, State: Initialized, Role: FOLLOWER
I20260809 22:52:23.281873 17436 consensus_queue.cc:260] T 904f38da63744f4a9073a2bb6a648427 P 2e993ca7f86e4c65becb4b7e21ca4454 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75717bd9f1d14b3496b152a93b132293" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 38889 } } peers { permanent_uuid: "c2e8db58047a476fa02950aac0f295f7" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 44319 } } peers { permanent_uuid: "2e993ca7f86e4c65becb4b7e21ca4454" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 34661 } }
I20260809 22:52:23.282941 17408 heartbeater.cc:503] Master 127.15.113.190:44473 was elected leader, sending a full tablet report...
I20260809 22:52:23.283063 17432 raft_consensus.cc:348] T 904f38da63744f4a9073a2bb6a648427 P c2e8db58047a476fa02950aac0f295f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75717bd9f1d14b3496b152a93b132293" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 38889 } } peers { permanent_uuid: "c2e8db58047a476fa02950aac0f295f7" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 44319 } } peers { permanent_uuid: "2e993ca7f86e4c65becb4b7e21ca4454" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 34661 } }
I20260809 22:52:23.283454 17432 raft_consensus.cc:374] T 904f38da63744f4a9073a2bb6a648427 P c2e8db58047a476fa02950aac0f295f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:52:23.283548 17432 raft_consensus.cc:729] T 904f38da63744f4a9073a2bb6a648427 P c2e8db58047a476fa02950aac0f295f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2e8db58047a476fa02950aac0f295f7, State: Initialized, Role: FOLLOWER
I20260809 22:52:23.283607 17436 ts_tablet_manager.cc:1428] T 904f38da63744f4a9073a2bb6a648427 P 2e993ca7f86e4c65becb4b7e21ca4454: Time spent starting tablet: real 0.011s user 0.003s sys 0.009s
I20260809 22:52:23.284112 17432 consensus_queue.cc:260] T 904f38da63744f4a9073a2bb6a648427 P c2e8db58047a476fa02950aac0f295f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75717bd9f1d14b3496b152a93b132293" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 38889 } } peers { permanent_uuid: "c2e8db58047a476fa02950aac0f295f7" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 44319 } } peers { permanent_uuid: "2e993ca7f86e4c65becb4b7e21ca4454" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 34661 } }
I20260809 22:52:23.285774 17432 ts_tablet_manager.cc:1428] T 904f38da63744f4a9073a2bb6a648427 P c2e8db58047a476fa02950aac0f295f7: Time spent starting tablet: real 0.013s user 0.012s sys 0.001s
W20260809 22:52:23.405160 17409 tablet.cc:2369] T 904f38da63744f4a9073a2bb6a648427 P 2e993ca7f86e4c65becb4b7e21ca4454: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 22:52:23.424831 17199 tablet.cc:2369] T 904f38da63744f4a9073a2bb6a648427 P 75717bd9f1d14b3496b152a93b132293: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 22:52:23.441959 17000 tablet.cc:2369] T 904f38da63744f4a9073a2bb6a648427 P c2e8db58047a476fa02950aac0f295f7: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 22:52:23.614127 17440 raft_consensus.cc:482] T 904f38da63744f4a9073a2bb6a648427 P 2e993ca7f86e4c65becb4b7e21ca4454 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 22:52:23.614418 17440 raft_consensus.cc:504] T 904f38da63744f4a9073a2bb6a648427 P 2e993ca7f86e4c65becb4b7e21ca4454 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75717bd9f1d14b3496b152a93b132293" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 38889 } } peers { permanent_uuid: "c2e8db58047a476fa02950aac0f295f7" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 44319 } } peers { permanent_uuid: "2e993ca7f86e4c65becb4b7e21ca4454" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 34661 } }
I20260809 22:52:23.616001 17440 leader_election.cc:288] T 904f38da63744f4a9073a2bb6a648427 P 2e993ca7f86e4c65becb4b7e21ca4454 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 75717bd9f1d14b3496b152a93b132293 (127.15.113.130:38889), c2e8db58047a476fa02950aac0f295f7 (127.15.113.129:44319)
I20260809 22:52:23.623385 17427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:52:23.616626 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.113.131:38405 (local address 127.15.113.129:44319)
0809 22:52:23.616822 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0809 22:52:23.616828 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:52:23.616847 (+ 19us) server_negotiation.cc:407] Connection header received
0809 22:52:23.617157 (+ 310us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:52:23.617163 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:52:23.617249 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:52:23.617338 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:52:23.617969 (+ 631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:52:23.619001 (+ 1032us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:52:23.621421 (+ 2420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:52:23.622612 (+ 1191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:52:23.622782 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:52:23.622858 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0809 22:52:23.623082 (+ 224us) server_negotiation.cc:299] Negotiation successful
0809 22:52:23.623190 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260809 22:52:23.623596 17410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:52:23.616347 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.129:44319 (local address 127.15.113.131:38405)
0809 22:52:23.616778 (+ 431us) negotiation.cc:107] Waiting for socket to connect
0809 22:52:23.616798 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 22:52:23.616944 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:52:23.617533 (+ 589us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:52:23.617538 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:52:23.617611 (+ 73us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:52:23.617809 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:52:23.617818 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:52:23.619132 (+ 1314us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:52:23.619136 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:52:23.621256 (+ 2120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:52:23.621272 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:52:23.622763 (+ 1491us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:52:23.622768 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:52:23.622907 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:52:23.622941 (+ 34us) client_negotiation.cc:769] Sending connection context
0809 22:52:23.623044 (+ 103us) client_negotiation.cc:240] Negotiation successful
0809 22:52:23.623237 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":309,"mutex_wait_us":79,"spinlock_wait_cycles":23808}
I20260809 22:52:23.623772 17422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:52:23.616877 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.113.131:50371 (local address 127.15.113.130:38889)
0809 22:52:23.617101 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0809 22:52:23.617107 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:52:23.617477 (+ 370us) server_negotiation.cc:407] Connection header received
0809 22:52:23.617774 (+ 297us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:52:23.617778 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:52:23.617845 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:52:23.617939 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:52:23.618403 (+ 464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:52:23.619317 (+ 914us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:52:23.621637 (+ 2320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:52:23.622828 (+ 1191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:52:23.622961 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:52:23.623008 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0809 22:52:23.623406 (+ 398us) server_negotiation.cc:299] Negotiation successful
0809 22:52:23.623538 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":99}
I20260809 22:52:23.623869 17443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:52:23.616382 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.130:38889 (local address 127.15.113.131:50371)
0809 22:52:23.617382 (+ 1000us) negotiation.cc:107] Waiting for socket to connect
0809 22:52:23.617401 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 22:52:23.617573 (+ 172us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:52:23.617963 (+ 390us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:52:23.617966 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:52:23.617987 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:52:23.618259 (+ 272us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:52:23.618266 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:52:23.619456 (+ 1190us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:52:23.619462 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:52:23.621488 (+ 2026us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:52:23.621498 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:52:23.623011 (+ 1513us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:52:23.623017 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:52:23.623241 (+ 224us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:52:23.623273 (+ 32us) client_negotiation.cc:769] Sending connection context
0809 22:52:23.623365 (+ 92us) client_negotiation.cc:240] Negotiation successful
0809 22:52:23.623478 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":885,"mutex_wait_us":44,"thread_start_us":706,"threads_started":1}
I20260809 22:52:23.624866 17134 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "904f38da63744f4a9073a2bb6a648427" candidate_uuid: "2e993ca7f86e4c65becb4b7e21ca4454" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75717bd9f1d14b3496b152a93b132293" is_pre_election: true
I20260809 22:52:23.624861 16929 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "904f38da63744f4a9073a2bb6a648427" candidate_uuid: "2e993ca7f86e4c65becb4b7e21ca4454" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2e8db58047a476fa02950aac0f295f7" is_pre_election: true
I20260809 22:52:23.625388 17134 raft_consensus.cc:2454] T 904f38da63744f4a9073a2bb6a648427 P 75717bd9f1d14b3496b152a93b132293 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2e993ca7f86e4c65becb4b7e21ca4454 in term 0.
I20260809 22:52:23.625387 16929 raft_consensus.cc:2454] T 904f38da63744f4a9073a2bb6a648427 P c2e8db58047a476fa02950aac0f295f7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2e993ca7f86e4c65becb4b7e21ca4454 in term 0.
I20260809 22:52:23.626011 17231 leader_election.cc:302] T 904f38da63744f4a9073a2bb6a648427 P 2e993ca7f86e4c65becb4b7e21ca4454 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2e993ca7f86e4c65becb4b7e21ca4454, c2e8db58047a476fa02950aac0f295f7; no voters:
I20260809 22:52:23.626372 17440 raft_consensus.cc:2781] T 904f38da63744f4a9073a2bb6a648427 P 2e993ca7f86e4c65becb4b7e21ca4454 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 22:52:23.626523 17440 raft_consensus.cc:482] T 904f38da63744f4a9073a2bb6a648427 P 2e993ca7f86e4c65becb4b7e21ca4454 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 22:52:23.626622 17440 raft_consensus.cc:3037] T 904f38da63744f4a9073a2bb6a648427 P 2e993ca7f86e4c65becb4b7e21ca4454 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:52:23.628768 17440 raft_consensus.cc:504] T 904f38da63744f4a9073a2bb6a648427 P 2e993ca7f86e4c65becb4b7e21ca4454 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75717bd9f1d14b3496b152a93b132293" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 38889 } } peers { permanent_uuid: "c2e8db58047a476fa02950aac0f295f7" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 44319 } } peers { permanent_uuid: "2e993ca7f86e4c65becb4b7e21ca4454" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 34661 } }
I20260809 22:52:23.629362 17440 leader_election.cc:288] T 904f38da63744f4a9073a2bb6a648427 P 2e993ca7f86e4c65becb4b7e21ca4454 [CANDIDATE]: Term 1 election: Requested vote from peers 75717bd9f1d14b3496b152a93b132293 (127.15.113.130:38889), c2e8db58047a476fa02950aac0f295f7 (127.15.113.129:44319)
I20260809 22:52:23.629779 17134 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "904f38da63744f4a9073a2bb6a648427" candidate_uuid: "2e993ca7f86e4c65becb4b7e21ca4454" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75717bd9f1d14b3496b152a93b132293"
I20260809 22:52:23.629830 16929 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "904f38da63744f4a9073a2bb6a648427" candidate_uuid: "2e993ca7f86e4c65becb4b7e21ca4454" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2e8db58047a476fa02950aac0f295f7"
I20260809 22:52:23.630023 17134 raft_consensus.cc:3037] T 904f38da63744f4a9073a2bb6a648427 P 75717bd9f1d14b3496b152a93b132293 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:52:23.630036 16929 raft_consensus.cc:3037] T 904f38da63744f4a9073a2bb6a648427 P c2e8db58047a476fa02950aac0f295f7 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:52:23.633246 17134 raft_consensus.cc:2454] T 904f38da63744f4a9073a2bb6a648427 P 75717bd9f1d14b3496b152a93b132293 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2e993ca7f86e4c65becb4b7e21ca4454 in term 1.
I20260809 22:52:23.633246 16929 raft_consensus.cc:2454] T 904f38da63744f4a9073a2bb6a648427 P c2e8db58047a476fa02950aac0f295f7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2e993ca7f86e4c65becb4b7e21ca4454 in term 1.
I20260809 22:52:23.633626 17231 leader_election.cc:302] T 904f38da63744f4a9073a2bb6a648427 P 2e993ca7f86e4c65becb4b7e21ca4454 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2e993ca7f86e4c65becb4b7e21ca4454, c2e8db58047a476fa02950aac0f295f7; no voters:
I20260809 22:52:23.633924 17440 raft_consensus.cc:2781] T 904f38da63744f4a9073a2bb6a648427 P 2e993ca7f86e4c65becb4b7e21ca4454 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 22:52:23.634357 17440 raft_consensus.cc:686] T 904f38da63744f4a9073a2bb6a648427 P 2e993ca7f86e4c65becb4b7e21ca4454 [term 1 LEADER]: Becoming Leader. State: Replica: 2e993ca7f86e4c65becb4b7e21ca4454, State: Running, Role: LEADER
I20260809 22:52:23.634877 17440 consensus_queue.cc:237] T 904f38da63744f4a9073a2bb6a648427 P 2e993ca7f86e4c65becb4b7e21ca4454 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75717bd9f1d14b3496b152a93b132293" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 38889 } } peers { permanent_uuid: "c2e8db58047a476fa02950aac0f295f7" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 44319 } } peers { permanent_uuid: "2e993ca7f86e4c65becb4b7e21ca4454" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 34661 } }
I20260809 22:52:23.639436 16710 catalog_manager.cc:5495] T 904f38da63744f4a9073a2bb6a648427 P 2e993ca7f86e4c65becb4b7e21ca4454 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2e993ca7f86e4c65becb4b7e21ca4454 (127.15.113.131). New cstate: current_term: 1 leader_uuid: "2e993ca7f86e4c65becb4b7e21ca4454" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75717bd9f1d14b3496b152a93b132293" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 38889 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c2e8db58047a476fa02950aac0f295f7" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 44319 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2e993ca7f86e4c65becb4b7e21ca4454" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 34661 } health_report { overall_health: HEALTHY } } }
Aug 09 22:52:23 dist-test-slave-ncp9 krb5kdc[16633](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315941, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.113.129@KRBTEST.COM
I20260809 22:52:23.693868 17427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:52:23.678522 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40658 (local address 127.15.113.129:44319)
0809 22:52:23.678738 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0809 22:52:23.678745 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:52:23.679260 (+ 515us) server_negotiation.cc:407] Connection header received
0809 22:52:23.679426 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:52:23.679430 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:52:23.679494 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:52:23.679569 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:52:23.681629 (+ 2060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:52:23.682394 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:52:23.684194 (+ 1800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:52:23.684714 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:52:23.684802 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:52:23.688536 (+ 3734us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:52:23.688550 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:52:23.688566 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:52:23.688601 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:52:23.691419 (+ 2818us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:52:23.691968 (+ 549us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:52:23.691975 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:52:23.691977 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:52:23.692106 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:52:23.692369 (+ 263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:52:23.692372 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:52:23.692374 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:52:23.692711 (+ 337us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:52:23.693036 (+ 325us) server_negotiation.cc:1035] Waiting for connection context
0809 22:52:23.693318 (+ 282us) server_negotiation.cc:299] Negotiation successful
0809 22:52:23.693522 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
Aug 09 22:52:23 dist-test-slave-ncp9 krb5kdc[16633](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315943, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 09 22:52:23 dist-test-slave-ncp9 krb5kdc[16633](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315943, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.15.113.129@KRBTEST.COM
I20260809 22:52:23.744163 17427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:52:23.731102 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40662 (local address 127.15.113.129:44319)
0809 22:52:23.731314 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0809 22:52:23.731321 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:52:23.731640 (+ 319us) server_negotiation.cc:407] Connection header received
0809 22:52:23.731816 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:52:23.731820 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:52:23.731886 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:52:23.731973 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:52:23.733301 (+ 1328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:52:23.734047 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:52:23.735314 (+ 1267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:52:23.735739 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:52:23.735851 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:52:23.739629 (+ 3778us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:52:23.739634 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:52:23.739638 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:52:23.739671 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:52:23.742130 (+ 2459us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:52:23.742677 (+ 547us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:52:23.742680 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:52:23.742683 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:52:23.742802 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:52:23.743060 (+ 258us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:52:23.743063 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:52:23.743065 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:52:23.743304 (+ 239us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:52:23.743506 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0809 22:52:23.743687 (+ 181us) server_negotiation.cc:299] Negotiation successful
0809 22:52:23.743854 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
Aug 09 22:52:23 dist-test-slave-ncp9 krb5kdc[16633](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315943, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 09 22:52:23 dist-test-slave-ncp9 krb5kdc[16633](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315943, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.113.129@KRBTEST.COM
I20260809 22:52:23.791245 17427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:52:23.778936 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40666 (local address 127.15.113.129:44319)
0809 22:52:23.779128 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0809 22:52:23.779134 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:52:23.779426 (+ 292us) server_negotiation.cc:407] Connection header received
0809 22:52:23.779591 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:52:23.779595 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:52:23.779673 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:52:23.779758 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:52:23.781095 (+ 1337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:52:23.781802 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:52:23.782989 (+ 1187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:52:23.783380 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:52:23.783468 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:52:23.787009 (+ 3541us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:52:23.787014 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:52:23.787018 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:52:23.787049 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:52:23.789271 (+ 2222us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:52:23.789826 (+ 555us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:52:23.789829 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:52:23.789831 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:52:23.789945 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:52:23.790219 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:52:23.790222 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:52:23.790223 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:52:23.790453 (+ 230us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:52:23.790640 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0809 22:52:23.790828 (+ 188us) server_negotiation.cc:299] Negotiation successful
0809 22:52:23.790991 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
W20260809 22:52:23.792865 16864 tablet_service.cc:2908] Rejecting scan request for tablet 904f38da63744f4a9073a2bb6a648427: Uninitialized: clean time has not yet been initialized
I20260809 22:52:23.795562 15814 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskItYPJR/build/asan/bin/kudu pid 16799
I20260809 22:52:23.806609 17427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:52:23.796408 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40680 (local address 127.15.113.129:44319)
0809 22:52:23.796586 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0809 22:52:23.796592 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:52:23.796845 (+ 253us) server_negotiation.cc:407] Connection header received
0809 22:52:23.797035 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:52:23.797039 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:52:23.797094 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:52:23.797175 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:52:23.798217 (+ 1042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:52:23.798838 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:52:23.799988 (+ 1150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:52:23.800392 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:52:23.800492 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:52:23.802508 (+ 2016us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:52:23.802513 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:52:23.802516 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:52:23.802542 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:52:23.804666 (+ 2124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:52:23.805253 (+ 587us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:52:23.805256 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:52:23.805258 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:52:23.805361 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:52:23.805609 (+ 248us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:52:23.805612 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:52:23.805614 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:52:23.805835 (+ 221us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:52:23.806030 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0809 22:52:23.806195 (+ 165us) server_negotiation.cc:299] Negotiation successful
0809 22:52:23.806335 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260809 22:52:23.807152 16993 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:52:23.893821 15814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskItYPJR/build/asan/bin/kudu with pid 16799
I20260809 22:52:23.917006 15814 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskItYPJR/build/asan/bin/kudu pid 17008
Aug 09 22:52:23 dist-test-slave-ncp9 krb5kdc[16633](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315943, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.113.130@KRBTEST.COM
I20260809 22:52:23.931200 17422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:52:23.917774 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42998 (local address 127.15.113.130:38889)
0809 22:52:23.917957 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0809 22:52:23.917963 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:52:23.917982 (+ 19us) server_negotiation.cc:407] Connection header received
0809 22:52:23.918167 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:52:23.918171 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:52:23.918235 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:52:23.918319 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:52:23.919501 (+ 1182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:52:23.920304 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:52:23.921628 (+ 1324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:52:23.922031 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:52:23.922139 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:52:23.925933 (+ 3794us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:52:23.925951 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:52:23.925972 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:52:23.926016 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:52:23.928519 (+ 2503us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:52:23.929227 (+ 708us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:52:23.929234 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:52:23.929236 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:52:23.929368 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:52:23.929730 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:52:23.929734 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:52:23.929736 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:52:23.930099 (+ 363us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:52:23.930383 (+ 284us) server_negotiation.cc:1035] Waiting for connection context
0809 22:52:23.930680 (+ 297us) server_negotiation.cc:299] Negotiation successful
0809 22:52:23.930885 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260809 22:52:23.931847 17191 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:52:24.016608 15814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskItYPJR/build/asan/bin/kudu with pid 17008
I20260809 22:52:24.027770 17410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:52:24.027311 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.129:44319 (local address 127.15.113.131:45561)
0809 22:52:24.027484 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0809 22:52:24.027595 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.113.129:44319: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260809 22:52:24.028043 17231 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.15.113.129:44319: connect: Connection refused (error 111)
I20260809 22:52:24.029426 17410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:52:24.029091 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.129:44319 (local address 127.15.113.131:35833)
0809 22:52:24.029237 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0809 22:52:24.029309 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.113.129:44319: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260809 22:52:24.029606 17231 consensus_peers.cc:487] T 904f38da63744f4a9073a2bb6a648427 P 2e993ca7f86e4c65becb4b7e21ca4454 -> Peer c2e8db58047a476fa02950aac0f295f7 (127.15.113.129:44319): Couldn't send request to peer c2e8db58047a476fa02950aac0f295f7. Status: Network error: Client connection negotiation failed: client connection to 127.15.113.129:44319: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260809 22:52:24.040787 15814 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskItYPJR/build/asan/bin/kudu pid 17204
Aug 09 22:52:24 dist-test-slave-ncp9 krb5kdc[16633](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315943, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.113.131@KRBTEST.COM
I20260809 22:52:24.054656 17469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:52:24.041521 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56832 (local address 127.15.113.131:34661)
0809 22:52:24.041962 (+ 441us) server_negotiation.cc:206] Beginning negotiation
0809 22:52:24.041966 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 22:52:24.041987 (+ 21us) server_negotiation.cc:407] Connection header received
0809 22:52:24.042039 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:52:24.042042 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:52:24.042103 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:52:24.042201 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:52:24.043343 (+ 1142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:52:24.044039 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:52:24.045283 (+ 1244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:52:24.045686 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:52:24.045774 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:52:24.049609 (+ 3835us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:52:24.049622 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:52:24.049638 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:52:24.049674 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:52:24.052298 (+ 2624us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:52:24.052877 (+ 579us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:52:24.052883 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:52:24.052886 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:52:24.053070 (+ 184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:52:24.053430 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:52:24.053433 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:52:24.053435 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:52:24.053746 (+ 311us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:52:24.053961 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0809 22:52:24.054178 (+ 217us) server_negotiation.cc:299] Negotiation successful
0809 22:52:24.054335 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":346,"thread_start_us":233,"threads_started":1}
I20260809 22:52:24.055294 17402 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:52:24.141106 15814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskItYPJR/build/asan/bin/kudu with pid 17204
I20260809 22:52:24.164201 15814 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskItYPJR/build/asan/bin/kudu pid 16657
I20260809 22:52:24.165340 16753 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:52:24.271572 15814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskItYPJR/build/asan/bin/kudu with pid 16657
2026-08-09T22:52:24Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (4445 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315935617919-15814-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 22:52:24 dist-test-slave-ncp9 krb5kdc[17476](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 22:52:24 dist-test-slave-ncp9 krb5kdc[17476](info): set up 2 sockets
Aug 09 22:52:24 dist-test-slave-ncp9 krb5kdc[17476](info): commencing operation
krb5kdc: starting...
W20260809 22:52:25.449527 15814 mini_kdc.cc:121] Time spent starting KDC: real 1.130s user 0.000s sys 0.021s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315935617919-15814-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315935617919-15814-0/krb5kdc/test-user.keytab.
Aug 09 22:52:25 dist-test-slave-ncp9 krb5kdc[17476](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315945, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T22:52:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T22:52:25Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.113.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.113.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.113.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.190.keytab.
Entry for principal kudu/127.15.113.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.190.keytab.
I20260809 22:52:25.634476 15814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskItYPJR/build/asan/bin/kudu
/tmp/dist-test-taskItYPJR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315935617919-15814-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315935617919-15814-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315935617919-15814-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315935617919-15814-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.113.190:38741
--webserver_interface=127.15.113.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.190.keytab
--principal=kudu/127.15.113.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.113.148:42473
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.113.190:38741
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315935617919-15814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315935617919-15814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315935617919-15814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:52:25.953240 17492 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:52:25.953614 17492 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:52:25.953730 17492 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:52:25.960594 17492 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 22:52:25.960683 17492 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260809 22:52:25.960731 17492 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:52:25.960788 17492 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:52:25.960843 17492 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 22:52:25.960918 17492 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 22:52:25.970309 17492 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.113.148:42473
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315935617919-15814-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315935617919-15814-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.113.190:38741
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.190.keytab
--principal=kudu/127.15.113.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.113.190:38741
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315935617919-15814-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.113.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315935617919-15814-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:52:25.971477 17492 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:52:25.972726 17492 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:52:25.984117 17500 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:52:25.984149 17503 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:52:25.984192 17499 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:52:25.984956 17492 server_base.cc:1034] running on GCE node
I20260809 22:52:25.985749 17492 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:52:25.987030 17492 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:52:25.988252 17492 hybrid_clock.cc:648] HybridClock initialized: now 1786315945988196 us; error 27 us; skew 500 ppm
I20260809 22:52:25.988564 17492 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:52:25 dist-test-slave-ncp9 krb5kdc[17476](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315945, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.113.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:52:25.991466 17492 init.cc:377] Logged in from keytab as kudu/127.15.113.190@KRBTEST.COM (short username kudu)
I20260809 22:52:25.992817 17492 webserver.cc:460] Webserver started at http://127.15.113.190:33401/ using document root <none> and password file <none>
I20260809 22:52:25.993484 17492 fs_manager.cc:359] Metadata directory not provided
I20260809 22:52:25.993597 17492 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:52:25.993898 17492 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:52:25.996791 17492 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315935617919-15814-0/minicluster-data/master-0/data/instance:
uuid: "152c29ffd1d34d12b577dbc76022feba"
format_stamp: "Formatted at 2026-08-09 22:52:25 on dist-test-slave-ncp9"
server_key: "0a035786938444dff247c6aa60499dea"
server_key_iv: "322a0d970bf0fba876e7444148d9a2a6"
server_key_version: "encryptionkey@0"
I20260809 22:52:25.997578 17492 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315935617919-15814-0/minicluster-data/master-0/wal/instance:
uuid: "152c29ffd1d34d12b577dbc76022feba"
format_stamp: "Formatted at 2026-08-09 22:52:25 on dist-test-slave-ncp9"
server_key: "0a035786938444dff247c6aa60499dea"
server_key_iv: "322a0d970bf0fba876e7444148d9a2a6"
server_key_version: "encryptionkey@0"
I20260809 22:52:26.002453 17492 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260809 22:52:26.005667 17513 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:52:26.006866 17492 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 22:52:26.007021 17492 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315935617919-15814-0/minicluster-data/master-0/data,/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315935617919-15814-0/minicluster-data/master-0/wal
uuid: "152c29ffd1d34d12b577dbc76022feba"
format_stamp: "Formatted at 2026-08-09 22:52:25 on dist-test-slave-ncp9"
server_key: "0a035786938444dff247c6aa60499dea"
server_key_iv: "322a0d970bf0fba876e7444148d9a2a6"
server_key_version: "encryptionkey@0"
I20260809 22:52:26.007217 17492 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315935617919-15814-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315935617919-15814-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315935617919-15814-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:52:26.031725 17492 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:52:26.035446 17492 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:52:26.035699 17492 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:52:26.056015 17492 rpc_server.cc:307] RPC server started. Bound to: 127.15.113.190:38741
I20260809 22:52:26.056051 17603 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.113.190:38741 every 8 connection(s)
I20260809 22:52:26.057288 17492 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315935617919-15814-0/minicluster-data/master-0/data/info.pb
I20260809 22:52:26.060170 15814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskItYPJR/build/asan/bin/kudu as pid 17492
I20260809 22:52:26.060416 15814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315935617919-15814-0/minicluster-data/master-0/wal/instance
I20260809 22:52:26.061316 15814 external_mini_cluster.cc:1428] Setting key 20297dacb9ae6ef5d86dec804a63b7c0
I20260809 22:52:26.061342 17604 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:52:26.075949 17604 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 152c29ffd1d34d12b577dbc76022feba: Bootstrap starting.
Aug 09 22:52:26 dist-test-slave-ncp9 krb5kdc[17476](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315945, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.113.190@KRBTEST.COM
I20260809 22:52:26.079967 17604 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 152c29ffd1d34d12b577dbc76022feba: Neither blocks nor log segments found. Creating new log.
I20260809 22:52:26.081403 17604 log.cc:824] T 00000000000000000000000000000000 P 152c29ffd1d34d12b577dbc76022feba: Log is configured to *not* fsync() on all Append() calls
I20260809 22:52:26.084784 17604 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 152c29ffd1d34d12b577dbc76022feba: No bootstrap required, opened a new log
I20260809 22:52:26.086094 17607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:52:26.063299 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42370 (local address 127.15.113.190:38741)
0809 22:52:26.064043 (+ 744us) server_negotiation.cc:206] Beginning negotiation
0809 22:52:26.064055 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 22:52:26.064105 (+ 50us) server_negotiation.cc:407] Connection header received
0809 22:52:26.066208 (+ 2103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:52:26.066236 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:52:26.066613 (+ 377us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:52:26.066923 (+ 310us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:52:26.068611 (+ 1688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:52:26.069919 (+ 1308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:52:26.071643 (+ 1724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:52:26.072338 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:52:26.072475 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:52:26.077472 (+ 4997us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:52:26.077489 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:52:26.077509 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:52:26.077563 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:52:26.081510 (+ 3947us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:52:26.082233 (+ 723us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:52:26.082239 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:52:26.082245 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:52:26.082423 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:52:26.082840 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:52:26.082845 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:52:26.082849 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:52:26.083538 (+ 689us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:52:26.083831 (+ 293us) server_negotiation.cc:1035] Waiting for connection context
0809 22:52:26.084225 (+ 394us) server_negotiation.cc:299] Negotiation successful
0809 22:52:26.084643 (+ 418us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":549,"thread_start_us":336,"threads_started":1}
I20260809 22:52:26.092845 17604 raft_consensus.cc:348] T 00000000000000000000000000000000 P 152c29ffd1d34d12b577dbc76022feba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "152c29ffd1d34d12b577dbc76022feba" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 38741 } }
I20260809 22:52:26.093245 17604 raft_consensus.cc:374] T 00000000000000000000000000000000 P 152c29ffd1d34d12b577dbc76022feba [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:52:26.093330 17604 raft_consensus.cc:729] T 00000000000000000000000000000000 P 152c29ffd1d34d12b577dbc76022feba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 152c29ffd1d34d12b577dbc76022feba, State: Initialized, Role: FOLLOWER
I20260809 22:52:26.093830 17604 consensus_queue.cc:260] T 00000000000000000000000000000000 P 152c29ffd1d34d12b577dbc76022feba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "152c29ffd1d34d12b577dbc76022feba" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 38741 } }
I20260809 22:52:26.094053 17604 raft_consensus.cc:388] T 00000000000000000000000000000000 P 152c29ffd1d34d12b577dbc76022feba [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 22:52:26.094182 17604 raft_consensus.cc:482] T 00000000000000000000000000000000 P 152c29ffd1d34d12b577dbc76022feba [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 22:52:26.094360 17604 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 152c29ffd1d34d12b577dbc76022feba [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:52:26.096649 17604 raft_consensus.cc:504] T 00000000000000000000000000000000 P 152c29ffd1d34d12b577dbc76022feba [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "152c29ffd1d34d12b577dbc76022feba" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 38741 } }
I20260809 22:52:26.097177 17604 leader_election.cc:302] T 00000000000000000000000000000000 P 152c29ffd1d34d12b577dbc76022feba [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 152c29ffd1d34d12b577dbc76022feba; no voters:
I20260809 22:52:26.097700 17604 leader_election.cc:288] T 00000000000000000000000000000000 P 152c29ffd1d34d12b577dbc76022feba [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 22:52:26.097815 17609 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 152c29ffd1d34d12b577dbc76022feba [term 1 FOLLOWER]: Leader election won for term 1
I20260809 22:52:26.098505 17609 raft_consensus.cc:686] T 00000000000000000000000000000000 P 152c29ffd1d34d12b577dbc76022feba [term 1 LEADER]: Becoming Leader. State: Replica: 152c29ffd1d34d12b577dbc76022feba, State: Running, Role: LEADER
I20260809 22:52:26.099033 17609 consensus_queue.cc:237] T 00000000000000000000000000000000 P 152c29ffd1d34d12b577dbc76022feba [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "152c29ffd1d34d12b577dbc76022feba" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 38741 } }
I20260809 22:52:26.099903 17604 sys_catalog.cc:564] T 00000000000000000000000000000000 P 152c29ffd1d34d12b577dbc76022feba [sys.catalog]: configured and running, proceeding with master startup.
I20260809 22:52:26.102101 17611 sys_catalog.cc:455] T 00000000000000000000000000000000 P 152c29ffd1d34d12b577dbc76022feba [sys.catalog]: SysCatalogTable state changed. Reason: New leader 152c29ffd1d34d12b577dbc76022feba. Latest consensus state: current_term: 1 leader_uuid: "152c29ffd1d34d12b577dbc76022feba" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "152c29ffd1d34d12b577dbc76022feba" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 38741 } } }
I20260809 22:52:26.102077 17610 sys_catalog.cc:455] T 00000000000000000000000000000000 P 152c29ffd1d34d12b577dbc76022feba [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "152c29ffd1d34d12b577dbc76022feba" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "152c29ffd1d34d12b577dbc76022feba" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 38741 } } }
I20260809 22:52:26.102669 17611 sys_catalog.cc:458] T 00000000000000000000000000000000 P 152c29ffd1d34d12b577dbc76022feba [sys.catalog]: This master's current role is: LEADER
I20260809 22:52:26.102746 17610 sys_catalog.cc:458] T 00000000000000000000000000000000 P 152c29ffd1d34d12b577dbc76022feba [sys.catalog]: This master's current role is: LEADER
I20260809 22:52:26.103884 17617 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 22:52:26.108758 17617 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 22:52:26.117734 17617 catalog_manager.cc:1347] Generated new cluster ID: df429851a0de493798316881672016fa
I20260809 22:52:26.117843 17617 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 22:52:26.126192 17617 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 22:52:26.127346 17617 catalog_manager.cc:1504] Loading token signing keys...
I20260809 22:52:26.134650 17617 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 152c29ffd1d34d12b577dbc76022feba: Generated new TSK 0
I20260809 22:52:26.135555 17617 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.113.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.113.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.113.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.129.keytab.
Entry for principal kudu/127.15.113.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.129.keytab.
I20260809 22:52:26.195746 15814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskItYPJR/build/asan/bin/kudu
/tmp/dist-test-taskItYPJR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315935617919-15814-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315935617919-15814-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315935617919-15814-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315935617919-15814-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.113.129:0
--local_ip_for_outbound_sockets=127.15.113.129
--webserver_interface=127.15.113.129
--webserver_port=0
--tserver_master_addrs=127.15.113.190:38741
--keytab_file=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.129.keytab
--principal=kudu/127.15.113.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.113.148:42473
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315935617919-15814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315935617919-15814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315935617919-15814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:52:26.521868 17644 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:52:26.522135 17644 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:52:26.522261 17644 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:52:26.529248 17644 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:52:26.529345 17644 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:52:26.529433 17644 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 22:52:26.529511 17644 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.113.129
I20260809 22:52:26.538784 17644 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.113.148:42473
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315935617919-15814-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315935617919-15814-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.129.keytab
--principal=kudu/127.15.113.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.113.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315935617919-15814-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.113.129
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.15.113.190:38741
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.113.129
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315935617919-15814-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:52:26.539913 17644 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:52:26.541227 17644 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:52:26.553493 17655 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:52:26.553435 17658 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:52:26.553468 17656 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:52:26.554544 17644 server_base.cc:1034] running on GCE node
I20260809 22:52:26.555092 17644 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:52:26.555932 17644 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:52:26.557132 17644 hybrid_clock.cc:648] HybridClock initialized: now 1786315946557062 us; error 51 us; skew 500 ppm
I20260809 22:52:26.557404 17644 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:52:26 dist-test-slave-ncp9 krb5kdc[17476](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315946, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.113.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:52:26.560326 17644 init.cc:377] Logged in from keytab as kudu/127.15.113.129@KRBTEST.COM (short username kudu)
I20260809 22:52:26.561765 17644 webserver.cc:460] Webserver started at http://127.15.113.129:45301/ using document root <none> and password file <none>
I20260809 22:52:26.562458 17644 fs_manager.cc:359] Metadata directory not provided
I20260809 22:52:26.562557 17644 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:52:26.562878 17644 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:52:26.565840 17644 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315935617919-15814-0/minicluster-data/ts-0/data/instance:
uuid: "610b38e390064672bc4c03a90c4bb702"
format_stamp: "Formatted at 2026-08-09 22:52:26 on dist-test-slave-ncp9"
server_key: "3405e178ce525807fde61913c6857b6e"
server_key_iv: "42275d7abf73d299483227fd7a059d68"
server_key_version: "encryptionkey@0"
I20260809 22:52:26.566593 17644 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315935617919-15814-0/minicluster-data/ts-0/wal/instance:
uuid: "610b38e390064672bc4c03a90c4bb702"
format_stamp: "Formatted at 2026-08-09 22:52:26 on dist-test-slave-ncp9"
server_key: "3405e178ce525807fde61913c6857b6e"
server_key_iv: "42275d7abf73d299483227fd7a059d68"
server_key_version: "encryptionkey@0"
I20260809 22:52:26.571410 17644 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260809 22:52:26.574622 17669 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:52:26.575810 17644 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260809 22:52:26.575953 17644 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315935617919-15814-0/minicluster-data/ts-0/data,/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315935617919-15814-0/minicluster-data/ts-0/wal
uuid: "610b38e390064672bc4c03a90c4bb702"
format_stamp: "Formatted at 2026-08-09 22:52:26 on dist-test-slave-ncp9"
server_key: "3405e178ce525807fde61913c6857b6e"
server_key_iv: "42275d7abf73d299483227fd7a059d68"
server_key_version: "encryptionkey@0"
I20260809 22:52:26.576139 17644 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315935617919-15814-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315935617919-15814-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315935617919-15814-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:52:26.590484 17644 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:52:26.594017 17644 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:52:26.594250 17644 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:52:26.596697 17644 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:52:26.596781 17644 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:52:26.596923 17644 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:52:26.596993 17644 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.001s
Aug 09 22:52:26 dist-test-slave-ncp9 krb5kdc[17476](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315946, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.113.129@KRBTEST.COM for kudu/127.15.113.190@KRBTEST.COM
I20260809 22:52:26.619031 17688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:52:26.597370 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.113.129:51673 (local address 127.15.113.190:38741)
0809 22:52:26.599507 (+ 2137us) server_negotiation.cc:206] Beginning negotiation
0809 22:52:26.599514 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:52:26.599538 (+ 24us) server_negotiation.cc:407] Connection header received
0809 22:52:26.601156 (+ 1618us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:52:26.601162 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:52:26.601250 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:52:26.601345 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:52:26.603469 (+ 2124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:52:26.604464 (+ 995us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:52:26.606999 (+ 2535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:52:26.607681 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:52:26.607831 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:52:26.612264 (+ 4433us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:52:26.612272 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:52:26.612277 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:52:26.612320 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:52:26.615517 (+ 3197us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:52:26.616446 (+ 929us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:52:26.616453 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:52:26.616457 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:52:26.616616 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:52:26.617214 (+ 598us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:52:26.617219 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:52:26.617223 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:52:26.617546 (+ 323us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:52:26.617731 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0809 22:52:26.618581 (+ 850us) server_negotiation.cc:299] Negotiation successful
0809 22:52:26.618762 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2011,"thread_start_us":1901,"threads_started":1}
I20260809 22:52:26.620610 17689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:52:26.597876 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:38741 (local address 127.15.113.129:51673)
0809 22:52:26.598760 (+ 884us) negotiation.cc:107] Waiting for socket to connect
0809 22:52:26.598819 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0809 22:52:26.600834 (+ 2015us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:52:26.601617 (+ 783us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:52:26.601627 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:52:26.602418 (+ 791us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:52:26.603298 (+ 880us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:52:26.603318 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:52:26.604672 (+ 1354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:52:26.604676 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:52:26.606821 (+ 2145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:52:26.606840 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:52:26.607937 (+ 1097us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:52:26.607945 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:52:26.608248 (+ 303us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:52:26.609058 (+ 810us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:52:26.609092 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:52:26.612031 (+ 2939us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:52:26.615633 (+ 3602us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:52:26.615646 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:52:26.615662 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:52:26.616270 (+ 608us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:52:26.616732 (+ 462us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:52:26.616737 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:52:26.616741 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:52:26.617045 (+ 304us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:52:26.617696 (+ 651us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:52:26.617709 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:52:26.618180 (+ 471us) client_negotiation.cc:769] Sending connection context
0809 22:52:26.618500 (+ 320us) client_negotiation.cc:240] Negotiation successful
0809 22:52:26.619065 (+ 565us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":621,"thread_start_us":354,"threads_started":1}
I20260809 22:52:26.640100 17688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:52:26.626422 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.113.129:35849 (local address 127.15.113.190:38741)
0809 22:52:26.626605 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0809 22:52:26.626610 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:52:26.627206 (+ 596us) server_negotiation.cc:407] Connection header received
0809 22:52:26.627395 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:52:26.627401 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:52:26.627483 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:52:26.627588 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:52:26.628776 (+ 1188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:52:26.629736 (+ 960us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:52:26.631780 (+ 2044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:52:26.632226 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:52:26.632340 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:52:26.634647 (+ 2307us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:52:26.634652 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:52:26.634655 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:52:26.634685 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:52:26.637630 (+ 2945us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:52:26.638271 (+ 641us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:52:26.638277 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:52:26.638281 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:52:26.638408 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:52:26.638763 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:52:26.638768 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:52:26.638771 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:52:26.639021 (+ 250us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:52:26.639162 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0809 22:52:26.639548 (+ 386us) server_negotiation.cc:299] Negotiation successful
0809 22:52:26.639755 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260809 22:52:26.640193 17788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:52:26.626421 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:38741 (local address 127.15.113.129:35849)
0809 22:52:26.627080 (+ 659us) negotiation.cc:107] Waiting for socket to connect
0809 22:52:26.627102 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0809 22:52:26.627260 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:52:26.627637 (+ 377us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:52:26.627642 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:52:26.628261 (+ 619us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:52:26.628644 (+ 383us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:52:26.628656 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:52:26.629906 (+ 1250us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:52:26.629913 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:52:26.631629 (+ 1716us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:52:26.631645 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:52:26.632439 (+ 794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:52:26.632446 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:52:26.632602 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:52:26.633288 (+ 686us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:52:26.633314 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:52:26.634418 (+ 1104us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:52:26.637755 (+ 3337us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:52:26.637760 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:52:26.637765 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:52:26.638155 (+ 390us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:52:26.638514 (+ 359us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:52:26.638520 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:52:26.638524 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:52:26.638658 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:52:26.639136 (+ 478us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:52:26.639141 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:52:26.639319 (+ 178us) client_negotiation.cc:769] Sending connection context
0809 22:52:26.639436 (+ 117us) client_negotiation.cc:240] Negotiation successful
0809 22:52:26.639756 (+ 320us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":488,"thread_start_us":324,"threads_started":1}
I20260809 22:52:26.649741 17853 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.113.129:45313 every 8 connection(s)
I20260809 22:52:26.649736 17644 rpc_server.cc:307] RPC server started. Bound to: 127.15.113.129:45313
I20260809 22:52:26.651437 17644 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315935617919-15814-0/minicluster-data/ts-0/data/info.pb
I20260809 22:52:26.653839 17854 heartbeater.cc:347] Connected to a master server at 127.15.113.190:38741
I20260809 22:52:26.654053 17854 heartbeater.cc:464] Registering TS with master...
I20260809 22:52:26.654718 17854 heartbeater.cc:511] Master 127.15.113.190:38741 requested a full tablet report, sending...
I20260809 22:52:26.655658 15814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskItYPJR/build/asan/bin/kudu as pid 17644
I20260809 22:52:26.655896 15814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315935617919-15814-0/minicluster-data/ts-0/wal/instance
I20260809 22:52:26.656577 15814 external_mini_cluster.cc:1428] Setting key 1e2fcb52e478722dd7cc3339ecaf5144
I20260809 22:52:26.656783 17540 ts_manager.cc:194] Registered new tserver with Master: 610b38e390064672bc4c03a90c4bb702 (127.15.113.129:45313)
I20260809 22:52:26.658550 17540 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.113.129@KRBTEST.COM'} at 127.15.113.129:51673
WARNING: no policy specified for kudu/127.15.113.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.113.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.113.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.130.keytab.
Entry for principal kudu/127.15.113.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.130.keytab.
I20260809 22:52:26.701426 15814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskItYPJR/build/asan/bin/kudu
/tmp/dist-test-taskItYPJR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315935617919-15814-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315935617919-15814-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315935617919-15814-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315935617919-15814-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.113.130:0
--local_ip_for_outbound_sockets=127.15.113.130
--webserver_interface=127.15.113.130
--webserver_port=0
--tserver_master_addrs=127.15.113.190:38741
--keytab_file=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.130.keytab
--principal=kudu/127.15.113.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.113.148:42473
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315935617919-15814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315935617919-15814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315935617919-15814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:52:27.020040 17862 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:52:27.020308 17862 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:52:27.020455 17862 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:52:27.027518 17862 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:52:27.027631 17862 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:52:27.027755 17862 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 22:52:27.027863 17862 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.113.130
I20260809 22:52:27.037194 17862 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.113.148:42473
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315935617919-15814-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315935617919-15814-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.130.keytab
--principal=kudu/127.15.113.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.113.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315935617919-15814-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.113.130
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.15.113.190:38741
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.113.130
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315935617919-15814-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:52:27.038369 17862 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:52:27.039765 17862 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:52:27.052204 17868 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:52:27.052112 17872 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:52:27.052246 17869 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:52:27.053246 17862 server_base.cc:1034] running on GCE node
I20260809 22:52:27.053829 17862 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:52:27.054677 17862 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:52:27.055919 17862 hybrid_clock.cc:648] HybridClock initialized: now 1786315947055835 us; error 49 us; skew 500 ppm
I20260809 22:52:27.056269 17862 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:52:27 dist-test-slave-ncp9 krb5kdc[17476](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315947, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.113.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:52:27.059896 17862 init.cc:377] Logged in from keytab as kudu/127.15.113.130@KRBTEST.COM (short username kudu)
I20260809 22:52:27.070658 17862 webserver.cc:460] Webserver started at http://127.15.113.130:46633/ using document root <none> and password file <none>
I20260809 22:52:27.071328 17862 fs_manager.cc:359] Metadata directory not provided
I20260809 22:52:27.071434 17862 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:52:27.071745 17862 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:52:27.074793 17862 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315935617919-15814-0/minicluster-data/ts-1/data/instance:
uuid: "7cc5e8e5db8840fda0686859e4c61823"
format_stamp: "Formatted at 2026-08-09 22:52:27 on dist-test-slave-ncp9"
server_key: "8657b4c4fed63001180a2dbb0191ad0e"
server_key_iv: "431893e88fff667a4cd725da167e535b"
server_key_version: "encryptionkey@0"
I20260809 22:52:27.075558 17862 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315935617919-15814-0/minicluster-data/ts-1/wal/instance:
uuid: "7cc5e8e5db8840fda0686859e4c61823"
format_stamp: "Formatted at 2026-08-09 22:52:27 on dist-test-slave-ncp9"
server_key: "8657b4c4fed63001180a2dbb0191ad0e"
server_key_iv: "431893e88fff667a4cd725da167e535b"
server_key_version: "encryptionkey@0"
I20260809 22:52:27.080521 17862 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.004s
I20260809 22:52:27.083807 17883 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:52:27.085111 17862 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 22:52:27.085260 17862 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315935617919-15814-0/minicluster-data/ts-1/data,/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315935617919-15814-0/minicluster-data/ts-1/wal
uuid: "7cc5e8e5db8840fda0686859e4c61823"
format_stamp: "Formatted at 2026-08-09 22:52:27 on dist-test-slave-ncp9"
server_key: "8657b4c4fed63001180a2dbb0191ad0e"
server_key_iv: "431893e88fff667a4cd725da167e535b"
server_key_version: "encryptionkey@0"
I20260809 22:52:27.085451 17862 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315935617919-15814-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315935617919-15814-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315935617919-15814-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:52:27.098049 17862 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:52:27.101879 17862 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:52:27.102111 17862 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:52:27.104485 17862 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:52:27.104566 17862 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:52:27.104696 17862 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:52:27.104769 17862 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 22:52:27 dist-test-slave-ncp9 krb5kdc[17476](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315947, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.113.130@KRBTEST.COM for kudu/127.15.113.190@KRBTEST.COM
I20260809 22:52:27.126273 17688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:52:27.105060 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.113.130:48097 (local address 127.15.113.190:38741)
0809 22:52:27.105257 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0809 22:52:27.105263 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:52:27.106522 (+ 1259us) server_negotiation.cc:407] Connection header received
0809 22:52:27.108808 (+ 2286us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:52:27.108814 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:52:27.108924 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:52:27.108994 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:52:27.110681 (+ 1687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:52:27.111846 (+ 1165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:52:27.114261 (+ 2415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:52:27.114899 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:52:27.115037 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:52:27.119408 (+ 4371us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:52:27.119413 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:52:27.119416 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:52:27.119447 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:52:27.122611 (+ 3164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:52:27.123568 (+ 957us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:52:27.123574 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:52:27.123577 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:52:27.123773 (+ 196us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:52:27.124360 (+ 587us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:52:27.124365 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:52:27.124368 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:52:27.124695 (+ 327us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:52:27.124836 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0809 22:52:27.125726 (+ 890us) server_negotiation.cc:299] Negotiation successful
0809 22:52:27.125944 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260809 22:52:27.127508 17898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:52:27.105492 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:38741 (local address 127.15.113.130:48097)
0809 22:52:27.106329 (+ 837us) negotiation.cc:107] Waiting for socket to connect
0809 22:52:27.106384 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0809 22:52:27.108415 (+ 2031us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:52:27.109388 (+ 973us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:52:27.109395 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:52:27.109848 (+ 453us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:52:27.110497 (+ 649us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:52:27.110520 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:52:27.112024 (+ 1504us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:52:27.112029 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:52:27.114078 (+ 2049us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:52:27.114090 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:52:27.115158 (+ 1068us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:52:27.115167 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:52:27.115361 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:52:27.115976 (+ 615us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:52:27.116010 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:52:27.119201 (+ 3191us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:52:27.122796 (+ 3595us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:52:27.122804 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:52:27.122818 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:52:27.123418 (+ 600us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:52:27.123905 (+ 487us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:52:27.123910 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:52:27.123914 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:52:27.124229 (+ 315us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:52:27.124843 (+ 614us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:52:27.124851 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:52:27.125265 (+ 414us) client_negotiation.cc:769] Sending connection context
0809 22:52:27.125602 (+ 337us) client_negotiation.cc:240] Negotiation successful
0809 22:52:27.126139 (+ 537us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":570,"thread_start_us":356,"threads_started":1}
I20260809 22:52:27.142213 17862 rpc_server.cc:307] RPC server started. Bound to: 127.15.113.130:37287
I20260809 22:52:27.142280 18077 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.113.130:37287 every 8 connection(s)
I20260809 22:52:27.143502 17862 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315935617919-15814-0/minicluster-data/ts-1/data/info.pb
I20260809 22:52:27.146377 18078 heartbeater.cc:347] Connected to a master server at 127.15.113.190:38741
I20260809 22:52:27.146615 18078 heartbeater.cc:464] Registering TS with master...
I20260809 22:52:27.147357 18078 heartbeater.cc:511] Master 127.15.113.190:38741 requested a full tablet report, sending...
I20260809 22:52:27.149158 17540 ts_manager.cc:194] Registered new tserver with Master: 7cc5e8e5db8840fda0686859e4c61823 (127.15.113.130:37287)
I20260809 22:52:27.150127 17540 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.113.130@KRBTEST.COM'} at 127.15.113.130:48097
I20260809 22:52:27.150987 15814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskItYPJR/build/asan/bin/kudu as pid 17862
I20260809 22:52:27.151188 15814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315935617919-15814-0/minicluster-data/ts-1/wal/instance
I20260809 22:52:27.151657 15814 external_mini_cluster.cc:1428] Setting key ac7d9eeed4fc1a2b322007912bbb8724
I20260809 22:52:27.154224 17688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:52:27.135178 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.113.130:37053 (local address 127.15.113.190:38741)
0809 22:52:27.135343 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0809 22:52:27.135349 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:52:27.141485 (+ 6136us) server_negotiation.cc:407] Connection header received
0809 22:52:27.141665 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:52:27.141670 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:52:27.141730 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:52:27.141807 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:52:27.142872 (+ 1065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:52:27.143603 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:52:27.145482 (+ 1879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:52:27.146100 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:52:27.146228 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:52:27.148131 (+ 1903us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:52:27.148136 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:52:27.148139 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:52:27.148179 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:52:27.151233 (+ 3054us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:52:27.151855 (+ 622us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:52:27.151861 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:52:27.151864 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:52:27.151998 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:52:27.152607 (+ 609us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:52:27.152612 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:52:27.152615 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:52:27.153171 (+ 556us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:52:27.153313 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0809 22:52:27.153826 (+ 513us) server_negotiation.cc:299] Negotiation successful
0809 22:52:27.153999 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":55}
I20260809 22:52:27.154429 18044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:52:27.135011 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:38741 (local address 127.15.113.130:37053)
0809 22:52:27.141370 (+ 6359us) negotiation.cc:107] Waiting for socket to connect
0809 22:52:27.141393 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0809 22:52:27.141539 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:52:27.141855 (+ 316us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:52:27.141860 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:52:27.142421 (+ 561us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:52:27.142736 (+ 315us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:52:27.142746 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:52:27.143734 (+ 988us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:52:27.143740 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:52:27.145327 (+ 1587us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:52:27.145344 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:52:27.146291 (+ 947us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:52:27.146297 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:52:27.146470 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:52:27.147129 (+ 659us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:52:27.147148 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:52:27.147945 (+ 797us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:52:27.151353 (+ 3408us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:52:27.151357 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:52:27.151359 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:52:27.151742 (+ 383us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:52:27.152106 (+ 364us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:52:27.152111 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:52:27.152114 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:52:27.152274 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:52:27.153376 (+ 1102us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:52:27.153381 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:52:27.153554 (+ 173us) client_negotiation.cc:769] Sending connection context
0809 22:52:27.153706 (+ 152us) client_negotiation.cc:240] Negotiation successful
0809 22:52:27.153995 (+ 289us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6204,"thread_start_us":6056,"threads_started":1}
WARNING: no policy specified for kudu/127.15.113.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.113.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.113.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.131.keytab.
Entry for principal kudu/127.15.113.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.131.keytab.
I20260809 22:52:27.193594 15814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskItYPJR/build/asan/bin/kudu
/tmp/dist-test-taskItYPJR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315935617919-15814-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315935617919-15814-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315935617919-15814-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315935617919-15814-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.113.131:0
--local_ip_for_outbound_sockets=127.15.113.131
--webserver_interface=127.15.113.131
--webserver_port=0
--tserver_master_addrs=127.15.113.190:38741
--keytab_file=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.131.keytab
--principal=kudu/127.15.113.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.113.148:42473
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315935617919-15814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315935617919-15814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315935617919-15814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:52:27.502627 18082 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:52:27.502887 18082 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:52:27.502985 18082 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:52:27.509918 18082 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:52:27.510017 18082 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:52:27.510099 18082 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 22:52:27.510216 18082 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.113.131
I20260809 22:52:27.519953 18082 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.113.148:42473
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315935617919-15814-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315935617919-15814-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.131.keytab
--principal=kudu/127.15.113.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.113.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315935617919-15814-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.113.131
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.15.113.190:38741
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.113.131
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315935617919-15814-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:52:27.521207 18082 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:52:27.522571 18082 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:52:27.534734 18088 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:52:27.534696 18090 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:52:27.534786 18092 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:52:27.535722 18082 server_base.cc:1034] running on GCE node
I20260809 22:52:27.536264 18082 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:52:27.537155 18082 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:52:27.538357 18082 hybrid_clock.cc:648] HybridClock initialized: now 1786315947538282 us; error 49 us; skew 500 ppm
I20260809 22:52:27.538619 18082 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:52:27 dist-test-slave-ncp9 krb5kdc[17476](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315947, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.113.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:52:27.541728 18082 init.cc:377] Logged in from keytab as kudu/127.15.113.131@KRBTEST.COM (short username kudu)
I20260809 22:52:27.543067 18082 webserver.cc:460] Webserver started at http://127.15.113.131:43751/ using document root <none> and password file <none>
I20260809 22:52:27.543725 18082 fs_manager.cc:359] Metadata directory not provided
I20260809 22:52:27.543826 18082 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:52:27.544135 18082 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:52:27.546947 18082 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315935617919-15814-0/minicluster-data/ts-2/data/instance:
uuid: "6d8a1e80214f4c0192bf6f3f3c7d5b19"
format_stamp: "Formatted at 2026-08-09 22:52:27 on dist-test-slave-ncp9"
server_key: "ae0a2ded4d96c545c96f8be637d12048"
server_key_iv: "e90a6bf32f201ef66b1495778133b93a"
server_key_version: "encryptionkey@0"
I20260809 22:52:27.547755 18082 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315935617919-15814-0/minicluster-data/ts-2/wal/instance:
uuid: "6d8a1e80214f4c0192bf6f3f3c7d5b19"
format_stamp: "Formatted at 2026-08-09 22:52:27 on dist-test-slave-ncp9"
server_key: "ae0a2ded4d96c545c96f8be637d12048"
server_key_iv: "e90a6bf32f201ef66b1495778133b93a"
server_key_version: "encryptionkey@0"
I20260809 22:52:27.552703 18082 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260809 22:52:27.555940 18099 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:52:27.557257 18082 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 22:52:27.557402 18082 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315935617919-15814-0/minicluster-data/ts-2/data,/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315935617919-15814-0/minicluster-data/ts-2/wal
uuid: "6d8a1e80214f4c0192bf6f3f3c7d5b19"
format_stamp: "Formatted at 2026-08-09 22:52:27 on dist-test-slave-ncp9"
server_key: "ae0a2ded4d96c545c96f8be637d12048"
server_key_iv: "e90a6bf32f201ef66b1495778133b93a"
server_key_version: "encryptionkey@0"
I20260809 22:52:27.557682 18082 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315935617919-15814-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315935617919-15814-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315935617919-15814-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:52:27.570667 18082 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:52:27.574609 18082 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:52:27.574838 18082 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:52:27.577212 18082 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:52:27.577288 18082 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:52:27.577399 18082 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:52:27.577463 18082 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 22:52:27 dist-test-slave-ncp9 krb5kdc[17476](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315947, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.113.131@KRBTEST.COM for kudu/127.15.113.190@KRBTEST.COM
I20260809 22:52:27.599342 17688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:52:27.577794 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.113.131:49237 (local address 127.15.113.190:38741)
0809 22:52:27.578002 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0809 22:52:27.578009 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:52:27.579385 (+ 1376us) server_negotiation.cc:407] Connection header received
0809 22:52:27.582068 (+ 2683us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:52:27.582074 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:52:27.582176 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:52:27.582306 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:52:27.584591 (+ 2285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:52:27.585601 (+ 1010us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:52:27.587857 (+ 2256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:52:27.588318 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:52:27.588422 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:52:27.592693 (+ 4271us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:52:27.592700 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:52:27.592705 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:52:27.592746 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:52:27.595673 (+ 2927us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:52:27.596689 (+ 1016us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:52:27.596694 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:52:27.596698 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:52:27.596831 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:52:27.597423 (+ 592us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:52:27.597426 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:52:27.597428 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:52:27.597687 (+ 259us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:52:27.597815 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0809 22:52:27.598732 (+ 917us) server_negotiation.cc:299] Negotiation successful
0809 22:52:27.598971 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260809 22:52:27.600586 18115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:52:27.578344 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:38741 (local address 127.15.113.131:49237)
0809 22:52:27.579205 (+ 861us) negotiation.cc:107] Waiting for socket to connect
0809 22:52:27.579260 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0809 22:52:27.581758 (+ 2498us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:52:27.582583 (+ 825us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:52:27.582594 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:52:27.583420 (+ 826us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:52:27.584381 (+ 961us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:52:27.584415 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:52:27.585770 (+ 1355us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:52:27.585774 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:52:27.587723 (+ 1949us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:52:27.587736 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:52:27.588559 (+ 823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:52:27.588567 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:52:27.588792 (+ 225us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:52:27.589476 (+ 684us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:52:27.589510 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:52:27.592481 (+ 2971us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:52:27.595815 (+ 3334us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:52:27.595825 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:52:27.595850 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:52:27.596550 (+ 700us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:52:27.596962 (+ 412us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:52:27.596967 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:52:27.596971 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:52:27.597264 (+ 293us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:52:27.597848 (+ 584us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:52:27.597856 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:52:27.598253 (+ 397us) client_negotiation.cc:769] Sending connection context
0809 22:52:27.598586 (+ 333us) client_negotiation.cc:240] Negotiation successful
0809 22:52:27.599145 (+ 559us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":579,"thread_start_us":315,"threads_started":1}
I20260809 22:52:27.621025 18082 rpc_server.cc:307] RPC server started. Bound to: 127.15.113.131:36465
I20260809 22:52:27.621062 18287 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.113.131:36465 every 8 connection(s)
I20260809 22:52:27.622558 18082 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315935617919-15814-0/minicluster-data/ts-2/data/info.pb
I20260809 22:52:27.623607 18251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:52:27.607046 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:38741 (local address 127.15.113.131:45959)
0809 22:52:27.607780 (+ 734us) negotiation.cc:107] Waiting for socket to connect
0809 22:52:27.607802 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0809 22:52:27.607948 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:52:27.608262 (+ 314us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:52:27.608267 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:52:27.608974 (+ 707us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:52:27.609366 (+ 392us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:52:27.609380 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:52:27.610354 (+ 974us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:52:27.610358 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:52:27.612029 (+ 1671us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:52:27.612045 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:52:27.612750 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:52:27.612756 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:52:27.612972 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:52:27.616238 (+ 3266us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:52:27.616263 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:52:27.617561 (+ 1298us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:52:27.620739 (+ 3178us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:52:27.620746 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:52:27.620754 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:52:27.621387 (+ 633us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:52:27.621725 (+ 338us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:52:27.621730 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:52:27.621733 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:52:27.621886 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:52:27.622354 (+ 468us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:52:27.622358 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:52:27.622555 (+ 197us) client_negotiation.cc:769] Sending connection context
0809 22:52:27.622703 (+ 148us) client_negotiation.cc:240] Negotiation successful
0809 22:52:27.623056 (+ 353us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":570,"thread_start_us":408,"threads_started":1}
I20260809 22:52:27.624351 17688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:52:27.607058 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.113.131:45959 (local address 127.15.113.190:38741)
0809 22:52:27.607243 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0809 22:52:27.607248 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:52:27.607890 (+ 642us) server_negotiation.cc:407] Connection header received
0809 22:52:27.608072 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:52:27.608076 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:52:27.608134 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:52:27.608218 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:52:27.609505 (+ 1287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:52:27.610227 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:52:27.612175 (+ 1948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:52:27.612580 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:52:27.612668 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:52:27.617776 (+ 5108us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:52:27.617783 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:52:27.617788 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:52:27.617825 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:52:27.620628 (+ 2803us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:52:27.621485 (+ 857us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:52:27.621489 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:52:27.621491 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:52:27.621610 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:52:27.621981 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:52:27.621984 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:52:27.621986 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:52:27.622257 (+ 271us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:52:27.622364 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0809 22:52:27.623847 (+ 1483us) server_negotiation.cc:299] Negotiation successful
0809 22:52:27.624079 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260809 22:52:27.624784 18289 heartbeater.cc:347] Connected to a master server at 127.15.113.190:38741
I20260809 22:52:27.625015 18289 heartbeater.cc:464] Registering TS with master...
I20260809 22:52:27.625771 18289 heartbeater.cc:511] Master 127.15.113.190:38741 requested a full tablet report, sending...
I20260809 22:52:27.627107 17538 ts_manager.cc:194] Registered new tserver with Master: 6d8a1e80214f4c0192bf6f3f3c7d5b19 (127.15.113.131:36465)
I20260809 22:52:27.627961 17538 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.113.131@KRBTEST.COM'} at 127.15.113.131:49237
I20260809 22:52:27.629112 15814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskItYPJR/build/asan/bin/kudu as pid 18082
I20260809 22:52:27.629304 15814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315935617919-15814-0/minicluster-data/ts-2/wal/instance
I20260809 22:52:27.629730 15814 external_mini_cluster.cc:1428] Setting key 842007c767bcef6fe345a1cc1dfb0a62
I20260809 22:52:27.632323 15814 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 22:52:27.661077 17854 heartbeater.cc:503] Master 127.15.113.190:38741 was elected leader, sending a full tablet report...
Aug 09 22:52:27 dist-test-slave-ncp9 krb5kdc[17476](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315947, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 09 22:52:27 dist-test-slave-ncp9 krb5kdc[17476](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315947, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.15.113.190@KRBTEST.COM
I20260809 22:52:27.681955 17688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:52:27.669269 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42384 (local address 127.15.113.190:38741)
0809 22:52:27.669501 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0809 22:52:27.669507 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:52:27.669719 (+ 212us) server_negotiation.cc:407] Connection header received
0809 22:52:27.669860 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:52:27.669864 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:52:27.669928 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:52:27.670007 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:52:27.671280 (+ 1273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:52:27.672018 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:52:27.673322 (+ 1304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:52:27.673774 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:52:27.673862 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:52:27.677322 (+ 3460us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:52:27.677327 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:52:27.677330 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:52:27.677361 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:52:27.679800 (+ 2439us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:52:27.680356 (+ 556us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:52:27.680360 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:52:27.680362 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:52:27.680487 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:52:27.680763 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:52:27.680766 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:52:27.680767 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:52:27.681045 (+ 278us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:52:27.681224 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0809 22:52:27.681471 (+ 247us) server_negotiation.cc:299] Negotiation successful
0809 22:52:27.681628 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260809 22:52:27.685668 17538 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:42384:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 22:52:27.692526 17538 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 22:52:27.718027 18307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:52:27.704921 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56810 (local address 127.15.113.131:36465)
0809 22:52:27.708574 (+ 3653us) server_negotiation.cc:206] Beginning negotiation
0809 22:52:27.708584 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 22:52:27.708619 (+ 35us) server_negotiation.cc:407] Connection header received
0809 22:52:27.708725 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:52:27.708734 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:52:27.709140 (+ 406us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:52:27.709339 (+ 199us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:52:27.710519 (+ 1180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:52:27.713186 (+ 2667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:52:27.715368 (+ 2182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:52:27.716821 (+ 1453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:52:27.717035 (+ 214us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:52:27.717226 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0809 22:52:27.717550 (+ 324us) server_negotiation.cc:299] Negotiation successful
0809 22:52:27.717719 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3459,"thread_start_us":2407,"threads_started":1}
I20260809 22:52:27.718065 18310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:52:27.704258 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.131:36465 (local address 127.0.0.1:56810)
0809 22:52:27.706499 (+ 2241us) negotiation.cc:107] Waiting for socket to connect
0809 22:52:27.706545 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0809 22:52:27.706746 (+ 201us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:52:27.709674 (+ 2928us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:52:27.709684 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:52:27.709719 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:52:27.710250 (+ 531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:52:27.710262 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:52:27.713375 (+ 3113us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:52:27.713381 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:52:27.715177 (+ 1796us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:52:27.715191 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:52:27.717043 (+ 1852us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:52:27.717050 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:52:27.717293 (+ 243us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:52:27.717365 (+ 72us) client_negotiation.cc:769] Sending connection context
0809 22:52:27.717483 (+ 118us) client_negotiation.cc:240] Negotiation successful
0809 22:52:27.717679 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1955,"mutex_wait_us":22,"thread_start_us":7043,"threads_started":1}
I20260809 22:52:27.720055 18316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:52:27.706864 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.129:45313 (local address 127.0.0.1:36642)
0809 22:52:27.712316 (+ 5452us) negotiation.cc:107] Waiting for socket to connect
0809 22:52:27.712338 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0809 22:52:27.712454 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:52:27.713230 (+ 776us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:52:27.713238 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:52:27.713264 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:52:27.713760 (+ 496us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:52:27.713773 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:52:27.715558 (+ 1785us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:52:27.715563 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:52:27.717339 (+ 1776us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:52:27.717353 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:52:27.719152 (+ 1799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:52:27.719158 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:52:27.719428 (+ 270us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:52:27.719477 (+ 49us) client_negotiation.cc:769] Sending connection context
0809 22:52:27.719563 (+ 86us) client_negotiation.cc:240] Negotiation successful
0809 22:52:27.719681 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5336,"mutex_wait_us":74,"thread_start_us":5184,"threads_started":1}
I20260809 22:52:27.720094 18308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:52:27.705068 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42956 (local address 127.15.113.130:37287)
0809 22:52:27.705878 (+ 810us) server_negotiation.cc:206] Beginning negotiation
0809 22:52:27.705888 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 22:52:27.711716 (+ 5828us) server_negotiation.cc:407] Connection header received
0809 22:52:27.712400 (+ 684us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:52:27.712408 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:52:27.712682 (+ 274us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:52:27.712946 (+ 264us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:52:27.713536 (+ 590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:52:27.714882 (+ 1346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:52:27.717346 (+ 2464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:52:27.718683 (+ 1337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:52:27.718857 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:52:27.719026 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0809 22:52:27.719654 (+ 628us) server_negotiation.cc:299] Negotiation successful
0809 22:52:27.719814 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":621,"thread_start_us":477,"threads_started":1}
I20260809 22:52:27.720095 18317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:52:27.707166 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36642 (local address 127.15.113.129:45313)
0809 22:52:27.711892 (+ 4726us) server_negotiation.cc:206] Beginning negotiation
0809 22:52:27.711901 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 22:52:27.712435 (+ 534us) server_negotiation.cc:407] Connection header received
0809 22:52:27.712575 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:52:27.712582 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:52:27.712786 (+ 204us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:52:27.713027 (+ 241us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:52:27.713973 (+ 946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:52:27.715393 (+ 1420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:52:27.717503 (+ 2110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:52:27.718929 (+ 1426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:52:27.719103 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:52:27.719251 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0809 22:52:27.719640 (+ 389us) server_negotiation.cc:299] Negotiation successful
0809 22:52:27.719818 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4559,"thread_start_us":4428,"threads_started":1}
I20260809 22:52:27.720757 18306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:52:27.705320 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.130:37287 (local address 127.0.0.1:42956)
0809 22:52:27.711584 (+ 6264us) negotiation.cc:107] Waiting for socket to connect
0809 22:52:27.711601 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 22:52:27.712224 (+ 623us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:52:27.712926 (+ 702us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:52:27.712931 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:52:27.712957 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:52:27.713361 (+ 404us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:52:27.713379 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:52:27.715024 (+ 1645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:52:27.715033 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:52:27.717200 (+ 2167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:52:27.717219 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:52:27.719199 (+ 1980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:52:27.719206 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:52:27.719444 (+ 238us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:52:27.719494 (+ 50us) client_negotiation.cc:769] Sending connection context
0809 22:52:27.720280 (+ 786us) client_negotiation.cc:240] Negotiation successful
0809 22:52:27.720404 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6143,"mutex_wait_us":53,"thread_start_us":724,"threads_started":1}
I20260809 22:52:27.722743 18178 tablet_service.cc:1467] Processing CreateTablet for tablet eae23c2246fa417a82a5ef0656420633 (DEFAULT_TABLE table=test-table [id=7a9c4d4047e8401ab08a24c180b2bb2f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:52:27.723644 17751 tablet_service.cc:1467] Processing CreateTablet for tablet eae23c2246fa417a82a5ef0656420633 (DEFAULT_TABLE table=test-table [id=7a9c4d4047e8401ab08a24c180b2bb2f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:52:27.724345 18178 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eae23c2246fa417a82a5ef0656420633. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:52:27.724217 17965 tablet_service.cc:1467] Processing CreateTablet for tablet eae23c2246fa417a82a5ef0656420633 (DEFAULT_TABLE table=test-table [id=7a9c4d4047e8401ab08a24c180b2bb2f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:52:27.724778 17751 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eae23c2246fa417a82a5ef0656420633. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:52:27.725560 17965 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eae23c2246fa417a82a5ef0656420633. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:52:27.737043 18318 tablet_bootstrap.cc:492] T eae23c2246fa417a82a5ef0656420633 P 7cc5e8e5db8840fda0686859e4c61823: Bootstrap starting.
I20260809 22:52:27.739504 18321 tablet_bootstrap.cc:492] T eae23c2246fa417a82a5ef0656420633 P 6d8a1e80214f4c0192bf6f3f3c7d5b19: Bootstrap starting.
I20260809 22:52:27.740460 18318 tablet_bootstrap.cc:654] T eae23c2246fa417a82a5ef0656420633 P 7cc5e8e5db8840fda0686859e4c61823: Neither blocks nor log segments found. Creating new log.
I20260809 22:52:27.740804 18320 tablet_bootstrap.cc:492] T eae23c2246fa417a82a5ef0656420633 P 610b38e390064672bc4c03a90c4bb702: Bootstrap starting.
I20260809 22:52:27.741901 18318 log.cc:824] T eae23c2246fa417a82a5ef0656420633 P 7cc5e8e5db8840fda0686859e4c61823: Log is configured to *not* fsync() on all Append() calls
I20260809 22:52:27.742905 18321 tablet_bootstrap.cc:654] T eae23c2246fa417a82a5ef0656420633 P 6d8a1e80214f4c0192bf6f3f3c7d5b19: Neither blocks nor log segments found. Creating new log.
I20260809 22:52:27.743835 18320 tablet_bootstrap.cc:654] T eae23c2246fa417a82a5ef0656420633 P 610b38e390064672bc4c03a90c4bb702: Neither blocks nor log segments found. Creating new log.
I20260809 22:52:27.744167 18321 log.cc:824] T eae23c2246fa417a82a5ef0656420633 P 6d8a1e80214f4c0192bf6f3f3c7d5b19: Log is configured to *not* fsync() on all Append() calls
I20260809 22:52:27.744800 18320 log.cc:824] T eae23c2246fa417a82a5ef0656420633 P 610b38e390064672bc4c03a90c4bb702: Log is configured to *not* fsync() on all Append() calls
I20260809 22:52:27.747347 18320 tablet_bootstrap.cc:492] T eae23c2246fa417a82a5ef0656420633 P 610b38e390064672bc4c03a90c4bb702: No bootstrap required, opened a new log
I20260809 22:52:27.747575 18320 ts_tablet_manager.cc:1397] T eae23c2246fa417a82a5ef0656420633 P 610b38e390064672bc4c03a90c4bb702: Time spent bootstrapping tablet: real 0.007s user 0.003s sys 0.002s
I20260809 22:52:27.747630 18321 tablet_bootstrap.cc:492] T eae23c2246fa417a82a5ef0656420633 P 6d8a1e80214f4c0192bf6f3f3c7d5b19: No bootstrap required, opened a new log
I20260809 22:52:27.747803 18318 tablet_bootstrap.cc:492] T eae23c2246fa417a82a5ef0656420633 P 7cc5e8e5db8840fda0686859e4c61823: No bootstrap required, opened a new log
I20260809 22:52:27.747891 18321 ts_tablet_manager.cc:1397] T eae23c2246fa417a82a5ef0656420633 P 6d8a1e80214f4c0192bf6f3f3c7d5b19: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260809 22:52:27.748029 18318 ts_tablet_manager.cc:1397] T eae23c2246fa417a82a5ef0656420633 P 7cc5e8e5db8840fda0686859e4c61823: Time spent bootstrapping tablet: real 0.011s user 0.005s sys 0.003s
I20260809 22:52:27.754738 18320 raft_consensus.cc:348] T eae23c2246fa417a82a5ef0656420633 P 610b38e390064672bc4c03a90c4bb702 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d8a1e80214f4c0192bf6f3f3c7d5b19" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 36465 } } peers { permanent_uuid: "7cc5e8e5db8840fda0686859e4c61823" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 37287 } } peers { permanent_uuid: "610b38e390064672bc4c03a90c4bb702" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 45313 } }
I20260809 22:52:27.755064 18320 raft_consensus.cc:374] T eae23c2246fa417a82a5ef0656420633 P 610b38e390064672bc4c03a90c4bb702 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:52:27.755157 18320 raft_consensus.cc:729] T eae23c2246fa417a82a5ef0656420633 P 610b38e390064672bc4c03a90c4bb702 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 610b38e390064672bc4c03a90c4bb702, State: Initialized, Role: FOLLOWER
I20260809 22:52:27.755703 18320 consensus_queue.cc:260] T eae23c2246fa417a82a5ef0656420633 P 610b38e390064672bc4c03a90c4bb702 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d8a1e80214f4c0192bf6f3f3c7d5b19" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 36465 } } peers { permanent_uuid: "7cc5e8e5db8840fda0686859e4c61823" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 37287 } } peers { permanent_uuid: "610b38e390064672bc4c03a90c4bb702" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 45313 } }
I20260809 22:52:27.757314 18320 ts_tablet_manager.cc:1428] T eae23c2246fa417a82a5ef0656420633 P 610b38e390064672bc4c03a90c4bb702: Time spent starting tablet: real 0.010s user 0.008s sys 0.003s
I20260809 22:52:27.758018 18318 raft_consensus.cc:348] T eae23c2246fa417a82a5ef0656420633 P 7cc5e8e5db8840fda0686859e4c61823 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d8a1e80214f4c0192bf6f3f3c7d5b19" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 36465 } } peers { permanent_uuid: "7cc5e8e5db8840fda0686859e4c61823" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 37287 } } peers { permanent_uuid: "610b38e390064672bc4c03a90c4bb702" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 45313 } }
I20260809 22:52:27.758462 18318 raft_consensus.cc:374] T eae23c2246fa417a82a5ef0656420633 P 7cc5e8e5db8840fda0686859e4c61823 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:52:27.758574 18318 raft_consensus.cc:729] T eae23c2246fa417a82a5ef0656420633 P 7cc5e8e5db8840fda0686859e4c61823 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7cc5e8e5db8840fda0686859e4c61823, State: Initialized, Role: FOLLOWER
I20260809 22:52:27.758533 18321 raft_consensus.cc:348] T eae23c2246fa417a82a5ef0656420633 P 6d8a1e80214f4c0192bf6f3f3c7d5b19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d8a1e80214f4c0192bf6f3f3c7d5b19" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 36465 } } peers { permanent_uuid: "7cc5e8e5db8840fda0686859e4c61823" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 37287 } } peers { permanent_uuid: "610b38e390064672bc4c03a90c4bb702" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 45313 } }
I20260809 22:52:27.759006 18321 raft_consensus.cc:374] T eae23c2246fa417a82a5ef0656420633 P 6d8a1e80214f4c0192bf6f3f3c7d5b19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:52:27.759112 18321 raft_consensus.cc:729] T eae23c2246fa417a82a5ef0656420633 P 6d8a1e80214f4c0192bf6f3f3c7d5b19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6d8a1e80214f4c0192bf6f3f3c7d5b19, State: Initialized, Role: FOLLOWER
I20260809 22:52:27.759187 18318 consensus_queue.cc:260] T eae23c2246fa417a82a5ef0656420633 P 7cc5e8e5db8840fda0686859e4c61823 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d8a1e80214f4c0192bf6f3f3c7d5b19" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 36465 } } peers { permanent_uuid: "7cc5e8e5db8840fda0686859e4c61823" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 37287 } } peers { permanent_uuid: "610b38e390064672bc4c03a90c4bb702" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 45313 } }
I20260809 22:52:27.759692 18321 consensus_queue.cc:260] T eae23c2246fa417a82a5ef0656420633 P 6d8a1e80214f4c0192bf6f3f3c7d5b19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d8a1e80214f4c0192bf6f3f3c7d5b19" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 36465 } } peers { permanent_uuid: "7cc5e8e5db8840fda0686859e4c61823" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 37287 } } peers { permanent_uuid: "610b38e390064672bc4c03a90c4bb702" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 45313 } }
I20260809 22:52:27.760423 18078 heartbeater.cc:503] Master 127.15.113.190:38741 was elected leader, sending a full tablet report...
I20260809 22:52:27.761024 18318 ts_tablet_manager.cc:1428] T eae23c2246fa417a82a5ef0656420633 P 7cc5e8e5db8840fda0686859e4c61823: Time spent starting tablet: real 0.013s user 0.013s sys 0.002s
I20260809 22:52:27.767246 18289 heartbeater.cc:503] Master 127.15.113.190:38741 was elected leader, sending a full tablet report...
I20260809 22:52:27.767918 18321 ts_tablet_manager.cc:1428] T eae23c2246fa417a82a5ef0656420633 P 6d8a1e80214f4c0192bf6f3f3c7d5b19: Time spent starting tablet: real 0.020s user 0.014s sys 0.004s
W20260809 22:52:27.874310 18290 tablet.cc:2369] T eae23c2246fa417a82a5ef0656420633 P 6d8a1e80214f4c0192bf6f3f3c7d5b19: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 22:52:27.894419 18326 raft_consensus.cc:482] T eae23c2246fa417a82a5ef0656420633 P 610b38e390064672bc4c03a90c4bb702 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 22:52:27.894752 18326 raft_consensus.cc:504] T eae23c2246fa417a82a5ef0656420633 P 610b38e390064672bc4c03a90c4bb702 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d8a1e80214f4c0192bf6f3f3c7d5b19" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 36465 } } peers { permanent_uuid: "7cc5e8e5db8840fda0686859e4c61823" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 37287 } } peers { permanent_uuid: "610b38e390064672bc4c03a90c4bb702" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 45313 } }
W20260809 22:52:27.895504 18079 tablet.cc:2369] T eae23c2246fa417a82a5ef0656420633 P 7cc5e8e5db8840fda0686859e4c61823: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 22:52:27.896577 18326 leader_election.cc:288] T eae23c2246fa417a82a5ef0656420633 P 610b38e390064672bc4c03a90c4bb702 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6d8a1e80214f4c0192bf6f3f3c7d5b19 (127.15.113.131:36465), 7cc5e8e5db8840fda0686859e4c61823 (127.15.113.130:37287)
W20260809 22:52:27.903784 17855 tablet.cc:2369] T eae23c2246fa417a82a5ef0656420633 P 610b38e390064672bc4c03a90c4bb702: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 22:52:27.905045 18307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:52:27.897138 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.113.129:50691 (local address 127.15.113.131:36465)
0809 22:52:27.897361 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0809 22:52:27.897367 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:52:27.898050 (+ 683us) server_negotiation.cc:407] Connection header received
0809 22:52:27.898198 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:52:27.898202 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:52:27.898266 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:52:27.898358 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:52:27.899020 (+ 662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:52:27.899917 (+ 897us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:52:27.902775 (+ 2858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:52:27.904020 (+ 1245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:52:27.904169 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:52:27.904235 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0809 22:52:27.904666 (+ 431us) server_negotiation.cc:299] Negotiation successful
0809 22:52:27.904785 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260809 22:52:27.905045 18308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:52:27.896994 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.113.129:51489 (local address 127.15.113.130:37287)
0809 22:52:27.897221 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0809 22:52:27.897228 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:52:27.897930 (+ 702us) server_negotiation.cc:407] Connection header received
0809 22:52:27.898123 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:52:27.898129 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:52:27.898208 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:52:27.898274 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:52:27.898955 (+ 681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:52:27.899860 (+ 905us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:52:27.902775 (+ 2915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:52:27.904002 (+ 1227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:52:27.904162 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:52:27.904235 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0809 22:52:27.904636 (+ 401us) server_negotiation.cc:299] Negotiation successful
0809 22:52:27.904768 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260809 22:52:27.905231 18338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:52:27.897172 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.131:36465 (local address 127.15.113.129:50691)
0809 22:52:27.897938 (+ 766us) negotiation.cc:107] Waiting for socket to connect
0809 22:52:27.897955 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 22:52:27.898082 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:52:27.898387 (+ 305us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:52:27.898392 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:52:27.898420 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:52:27.898860 (+ 440us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:52:27.898871 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:52:27.900107 (+ 1236us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:52:27.900112 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:52:27.902613 (+ 2501us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:52:27.902629 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:52:27.904399 (+ 1770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:52:27.904403 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:52:27.904519 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:52:27.904541 (+ 22us) client_negotiation.cc:769] Sending connection context
0809 22:52:27.904618 (+ 77us) client_negotiation.cc:240] Negotiation successful
0809 22:52:27.904749 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":641,"mutex_wait_us":95,"spinlock_wait_cycles":896,"thread_start_us":456,"threads_started":1}
I20260809 22:52:27.905275 18333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:52:27.896972 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.130:37287 (local address 127.15.113.129:51489)
0809 22:52:27.897792 (+ 820us) negotiation.cc:107] Waiting for socket to connect
0809 22:52:27.897812 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 22:52:27.897974 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:52:27.898311 (+ 337us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:52:27.898316 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:52:27.898341 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:52:27.898799 (+ 458us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:52:27.898810 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:52:27.899990 (+ 1180us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:52:27.899995 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:52:27.902627 (+ 2632us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:52:27.902637 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:52:27.904239 (+ 1602us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:52:27.904246 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:52:27.904410 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:52:27.904447 (+ 37us) client_negotiation.cc:769] Sending connection context
0809 22:52:27.904593 (+ 146us) client_negotiation.cc:240] Negotiation successful
0809 22:52:27.904730 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":663,"mutex_wait_us":115,"spinlock_wait_cycles":20864,"thread_start_us":421,"threads_started":1}
I20260809 22:52:27.906217 17992 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eae23c2246fa417a82a5ef0656420633" candidate_uuid: "610b38e390064672bc4c03a90c4bb702" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cc5e8e5db8840fda0686859e4c61823" is_pre_election: true
I20260809 22:52:27.906380 18212 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eae23c2246fa417a82a5ef0656420633" candidate_uuid: "610b38e390064672bc4c03a90c4bb702" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6d8a1e80214f4c0192bf6f3f3c7d5b19" is_pre_election: true
I20260809 22:52:27.906821 17992 raft_consensus.cc:2454] T eae23c2246fa417a82a5ef0656420633 P 7cc5e8e5db8840fda0686859e4c61823 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610b38e390064672bc4c03a90c4bb702 in term 0.
I20260809 22:52:27.906906 18212 raft_consensus.cc:2454] T eae23c2246fa417a82a5ef0656420633 P 6d8a1e80214f4c0192bf6f3f3c7d5b19 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610b38e390064672bc4c03a90c4bb702 in term 0.
I20260809 22:52:27.907577 17671 leader_election.cc:302] T eae23c2246fa417a82a5ef0656420633 P 610b38e390064672bc4c03a90c4bb702 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610b38e390064672bc4c03a90c4bb702, 6d8a1e80214f4c0192bf6f3f3c7d5b19; no voters:
I20260809 22:52:27.907971 18326 raft_consensus.cc:2781] T eae23c2246fa417a82a5ef0656420633 P 610b38e390064672bc4c03a90c4bb702 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 22:52:27.908138 18326 raft_consensus.cc:482] T eae23c2246fa417a82a5ef0656420633 P 610b38e390064672bc4c03a90c4bb702 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 22:52:27.908237 18326 raft_consensus.cc:3037] T eae23c2246fa417a82a5ef0656420633 P 610b38e390064672bc4c03a90c4bb702 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:52:27.910537 18326 raft_consensus.cc:504] T eae23c2246fa417a82a5ef0656420633 P 610b38e390064672bc4c03a90c4bb702 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d8a1e80214f4c0192bf6f3f3c7d5b19" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 36465 } } peers { permanent_uuid: "7cc5e8e5db8840fda0686859e4c61823" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 37287 } } peers { permanent_uuid: "610b38e390064672bc4c03a90c4bb702" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 45313 } }
I20260809 22:52:27.911083 18326 leader_election.cc:288] T eae23c2246fa417a82a5ef0656420633 P 610b38e390064672bc4c03a90c4bb702 [CANDIDATE]: Term 1 election: Requested vote from peers 6d8a1e80214f4c0192bf6f3f3c7d5b19 (127.15.113.131:36465), 7cc5e8e5db8840fda0686859e4c61823 (127.15.113.130:37287)
I20260809 22:52:27.911515 18212 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eae23c2246fa417a82a5ef0656420633" candidate_uuid: "610b38e390064672bc4c03a90c4bb702" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6d8a1e80214f4c0192bf6f3f3c7d5b19"
I20260809 22:52:27.911688 18212 raft_consensus.cc:3037] T eae23c2246fa417a82a5ef0656420633 P 6d8a1e80214f4c0192bf6f3f3c7d5b19 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:52:27.913079 17992 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eae23c2246fa417a82a5ef0656420633" candidate_uuid: "610b38e390064672bc4c03a90c4bb702" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cc5e8e5db8840fda0686859e4c61823"
I20260809 22:52:27.913260 17992 raft_consensus.cc:3037] T eae23c2246fa417a82a5ef0656420633 P 7cc5e8e5db8840fda0686859e4c61823 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:52:27.915337 17992 raft_consensus.cc:2454] T eae23c2246fa417a82a5ef0656420633 P 7cc5e8e5db8840fda0686859e4c61823 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610b38e390064672bc4c03a90c4bb702 in term 1.
I20260809 22:52:27.915721 17673 leader_election.cc:302] T eae23c2246fa417a82a5ef0656420633 P 610b38e390064672bc4c03a90c4bb702 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610b38e390064672bc4c03a90c4bb702, 7cc5e8e5db8840fda0686859e4c61823; no voters:
I20260809 22:52:27.916035 18326 raft_consensus.cc:2781] T eae23c2246fa417a82a5ef0656420633 P 610b38e390064672bc4c03a90c4bb702 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 22:52:27.916500 18326 raft_consensus.cc:686] T eae23c2246fa417a82a5ef0656420633 P 610b38e390064672bc4c03a90c4bb702 [term 1 LEADER]: Becoming Leader. State: Replica: 610b38e390064672bc4c03a90c4bb702, State: Running, Role: LEADER
I20260809 22:52:27.916656 18212 raft_consensus.cc:2454] T eae23c2246fa417a82a5ef0656420633 P 6d8a1e80214f4c0192bf6f3f3c7d5b19 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610b38e390064672bc4c03a90c4bb702 in term 1.
I20260809 22:52:27.917232 18326 consensus_queue.cc:237] T eae23c2246fa417a82a5ef0656420633 P 610b38e390064672bc4c03a90c4bb702 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d8a1e80214f4c0192bf6f3f3c7d5b19" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 36465 } } peers { permanent_uuid: "7cc5e8e5db8840fda0686859e4c61823" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 37287 } } peers { permanent_uuid: "610b38e390064672bc4c03a90c4bb702" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 45313 } }
I20260809 22:52:27.921895 17532 catalog_manager.cc:5495] T eae23c2246fa417a82a5ef0656420633 P 610b38e390064672bc4c03a90c4bb702 reported cstate change: term changed from 0 to 1, leader changed from <none> to 610b38e390064672bc4c03a90c4bb702 (127.15.113.129). New cstate: current_term: 1 leader_uuid: "610b38e390064672bc4c03a90c4bb702" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d8a1e80214f4c0192bf6f3f3c7d5b19" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 36465 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7cc5e8e5db8840fda0686859e4c61823" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 37287 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "610b38e390064672bc4c03a90c4bb702" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 45313 } health_report { overall_health: HEALTHY } } }
I20260809 22:52:27.962879 18317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:52:27.957835 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36652 (local address 127.15.113.129:45313)
0809 22:52:27.958011 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0809 22:52:27.958016 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:52:27.958209 (+ 193us) server_negotiation.cc:407] Connection header received
0809 22:52:27.958470 (+ 261us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:52:27.958473 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:52:27.958557 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:52:27.958671 (+ 114us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 22:52:27.958948 (+ 277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:52:27.959632 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:52:27.960908 (+ 1276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:52:27.961318 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:52:27.961407 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:52:27.961876 (+ 469us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 22:52:27.962387 (+ 511us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 22:52:27.962490 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0809 22:52:27.962615 (+ 125us) server_negotiation.cc:299] Negotiation successful
0809 22:52:27.962681 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260809 22:52:27.970739 17992 raft_consensus.cc:1261] T eae23c2246fa417a82a5ef0656420633 P 7cc5e8e5db8840fda0686859e4c61823 [term 1 FOLLOWER]: Refusing update from remote peer 610b38e390064672bc4c03a90c4bb702: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 22:52:27.970754 18212 raft_consensus.cc:1261] T eae23c2246fa417a82a5ef0656420633 P 6d8a1e80214f4c0192bf6f3f3c7d5b19 [term 1 FOLLOWER]: Refusing update from remote peer 610b38e390064672bc4c03a90c4bb702: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 22:52:27.971653 18326 consensus_queue.cc:1035] T eae23c2246fa417a82a5ef0656420633 P 610b38e390064672bc4c03a90c4bb702 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cc5e8e5db8840fda0686859e4c61823" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 37287 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 22:52:27.971930 18339 consensus_queue.cc:1035] T eae23c2246fa417a82a5ef0656420633 P 610b38e390064672bc4c03a90c4bb702 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6d8a1e80214f4c0192bf6f3f3c7d5b19" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 36465 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 22:52:27.999472 17540 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:42384:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 22:52:27.999748 17540 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:42384:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 22:52:28.003748 17540 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 152c29ffd1d34d12b577dbc76022feba: Sending DeleteTablet for 3 replicas of tablet eae23c2246fa417a82a5ef0656420633
I20260809 22:52:28.004994 18178 tablet_service.cc:1514] Processing DeleteTablet for tablet eae23c2246fa417a82a5ef0656420633 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:52:27 UTC) from {username='kudu', principal='kudu/127.15.113.190@KRBTEST.COM'} at 127.0.0.1:56810
I20260809 22:52:28.005111 17965 tablet_service.cc:1514] Processing DeleteTablet for tablet eae23c2246fa417a82a5ef0656420633 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:52:27 UTC) from {username='kudu', principal='kudu/127.15.113.190@KRBTEST.COM'} at 127.0.0.1:42956
I20260809 22:52:28.005739 17751 tablet_service.cc:1514] Processing DeleteTablet for tablet eae23c2246fa417a82a5ef0656420633 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:52:27 UTC) from {username='kudu', principal='kudu/127.15.113.190@KRBTEST.COM'} at 127.0.0.1:36642
I20260809 22:52:28.008673 17540 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:42384:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 22:52:28.010703 17540 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 22:52:28.013677 18362 tablet_replica.cc:330] T eae23c2246fa417a82a5ef0656420633 P 610b38e390064672bc4c03a90c4bb702: stopping tablet replica
I20260809 22:52:28.014771 18362 raft_consensus.cc:2229] T eae23c2246fa417a82a5ef0656420633 P 610b38e390064672bc4c03a90c4bb702 [term 1 LEADER]: Raft consensus shutting down.
I20260809 22:52:28.014961 18360 tablet_replica.cc:330] T eae23c2246fa417a82a5ef0656420633 P 6d8a1e80214f4c0192bf6f3f3c7d5b19: stopping tablet replica
I20260809 22:52:28.015385 18360 raft_consensus.cc:2229] T eae23c2246fa417a82a5ef0656420633 P 6d8a1e80214f4c0192bf6f3f3c7d5b19 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 22:52:28.015609 18362 raft_consensus.cc:2258] T eae23c2246fa417a82a5ef0656420633 P 610b38e390064672bc4c03a90c4bb702 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 22:52:28.016141 18360 raft_consensus.cc:2258] T eae23c2246fa417a82a5ef0656420633 P 6d8a1e80214f4c0192bf6f3f3c7d5b19 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 22:52:28.017833 18362 ts_tablet_manager.cc:1905] T eae23c2246fa417a82a5ef0656420633 P 610b38e390064672bc4c03a90c4bb702: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 22:52:28.021230 17751 tablet_service.cc:1467] Processing CreateTablet for tablet f61d736329a84372a0814ab03a66f100 (DEFAULT_TABLE table=test-table [id=8780eb4f4f234a7c955dcf149974b623]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:52:28.021486 17963 tablet_service.cc:1467] Processing CreateTablet for tablet f61d736329a84372a0814ab03a66f100 (DEFAULT_TABLE table=test-table [id=8780eb4f4f234a7c955dcf149974b623]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:52:28.021669 17751 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f61d736329a84372a0814ab03a66f100. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:52:28.022812 17963 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f61d736329a84372a0814ab03a66f100. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:52:28.025502 18178 tablet_service.cc:1467] Processing CreateTablet for tablet f61d736329a84372a0814ab03a66f100 (DEFAULT_TABLE table=test-table [id=8780eb4f4f234a7c955dcf149974b623]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:52:28.025924 18178 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f61d736329a84372a0814ab03a66f100. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:52:28.028985 18360 ts_tablet_manager.cc:1905] T eae23c2246fa417a82a5ef0656420633 P 6d8a1e80214f4c0192bf6f3f3c7d5b19: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 22:52:28.030113 18361 tablet_replica.cc:330] T eae23c2246fa417a82a5ef0656420633 P 7cc5e8e5db8840fda0686859e4c61823: stopping tablet replica
I20260809 22:52:28.031191 18318 tablet_bootstrap.cc:492] T f61d736329a84372a0814ab03a66f100 P 7cc5e8e5db8840fda0686859e4c61823: Bootstrap starting.
I20260809 22:52:28.032605 18318 tablet_bootstrap.cc:654] T f61d736329a84372a0814ab03a66f100 P 7cc5e8e5db8840fda0686859e4c61823: Neither blocks nor log segments found. Creating new log.
I20260809 22:52:28.032727 18321 tablet_bootstrap.cc:492] T f61d736329a84372a0814ab03a66f100 P 6d8a1e80214f4c0192bf6f3f3c7d5b19: Bootstrap starting.
I20260809 22:52:28.034853 18321 tablet_bootstrap.cc:654] T f61d736329a84372a0814ab03a66f100 P 6d8a1e80214f4c0192bf6f3f3c7d5b19: Neither blocks nor log segments found. Creating new log.
I20260809 22:52:28.036130 18320 tablet_bootstrap.cc:492] T f61d736329a84372a0814ab03a66f100 P 610b38e390064672bc4c03a90c4bb702: Bootstrap starting.
I20260809 22:52:28.036798 18360 ts_tablet_manager.cc:1918] T eae23c2246fa417a82a5ef0656420633 P 6d8a1e80214f4c0192bf6f3f3c7d5b19: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 22:52:28.036994 18360 log.cc:1196] T eae23c2246fa417a82a5ef0656420633 P 6d8a1e80214f4c0192bf6f3f3c7d5b19: Deleting WAL directory at /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315935617919-15814-0/minicluster-data/ts-2/wal/wals/eae23c2246fa417a82a5ef0656420633
I20260809 22:52:28.037570 18360 ts_tablet_manager.cc:1939] T eae23c2246fa417a82a5ef0656420633 P 6d8a1e80214f4c0192bf6f3f3c7d5b19: Deleting consensus metadata
I20260809 22:52:28.037683 18320 tablet_bootstrap.cc:654] T f61d736329a84372a0814ab03a66f100 P 610b38e390064672bc4c03a90c4bb702: Neither blocks nor log segments found. Creating new log.
I20260809 22:52:28.030556 18361 raft_consensus.cc:2229] T eae23c2246fa417a82a5ef0656420633 P 7cc5e8e5db8840fda0686859e4c61823 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 22:52:28.039529 17516 catalog_manager.cc:4841] TS 6d8a1e80214f4c0192bf6f3f3c7d5b19 (127.15.113.131:36465): tablet eae23c2246fa417a82a5ef0656420633 (table test-table [id=7a9c4d4047e8401ab08a24c180b2bb2f]) successfully deleted
I20260809 22:52:28.039846 18361 raft_consensus.cc:2258] T eae23c2246fa417a82a5ef0656420633 P 7cc5e8e5db8840fda0686859e4c61823 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 22:52:28.042223 18361 ts_tablet_manager.cc:1905] T eae23c2246fa417a82a5ef0656420633 P 7cc5e8e5db8840fda0686859e4c61823: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 22:52:28.043692 18362 ts_tablet_manager.cc:1918] T eae23c2246fa417a82a5ef0656420633 P 610b38e390064672bc4c03a90c4bb702: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 22:52:28.044216 18362 log.cc:1196] T eae23c2246fa417a82a5ef0656420633 P 610b38e390064672bc4c03a90c4bb702: Deleting WAL directory at /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315935617919-15814-0/minicluster-data/ts-0/wal/wals/eae23c2246fa417a82a5ef0656420633
I20260809 22:52:28.045001 18362 ts_tablet_manager.cc:1939] T eae23c2246fa417a82a5ef0656420633 P 610b38e390064672bc4c03a90c4bb702: Deleting consensus metadata
I20260809 22:52:28.047353 17519 catalog_manager.cc:4841] TS 610b38e390064672bc4c03a90c4bb702 (127.15.113.129:45313): tablet eae23c2246fa417a82a5ef0656420633 (table test-table [id=7a9c4d4047e8401ab08a24c180b2bb2f]) successfully deleted
I20260809 22:52:28.049252 18361 ts_tablet_manager.cc:1918] T eae23c2246fa417a82a5ef0656420633 P 7cc5e8e5db8840fda0686859e4c61823: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 22:52:28.049497 18361 log.cc:1196] T eae23c2246fa417a82a5ef0656420633 P 7cc5e8e5db8840fda0686859e4c61823: Deleting WAL directory at /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315935617919-15814-0/minicluster-data/ts-1/wal/wals/eae23c2246fa417a82a5ef0656420633
I20260809 22:52:28.050174 18361 ts_tablet_manager.cc:1939] T eae23c2246fa417a82a5ef0656420633 P 7cc5e8e5db8840fda0686859e4c61823: Deleting consensus metadata
I20260809 22:52:28.050961 18318 tablet_bootstrap.cc:492] T f61d736329a84372a0814ab03a66f100 P 7cc5e8e5db8840fda0686859e4c61823: No bootstrap required, opened a new log
I20260809 22:52:28.051208 18318 ts_tablet_manager.cc:1397] T f61d736329a84372a0814ab03a66f100 P 7cc5e8e5db8840fda0686859e4c61823: Time spent bootstrapping tablet: real 0.020s user 0.004s sys 0.006s
I20260809 22:52:28.051889 18320 tablet_bootstrap.cc:492] T f61d736329a84372a0814ab03a66f100 P 610b38e390064672bc4c03a90c4bb702: No bootstrap required, opened a new log
I20260809 22:52:28.052034 18320 ts_tablet_manager.cc:1397] T f61d736329a84372a0814ab03a66f100 P 610b38e390064672bc4c03a90c4bb702: Time spent bootstrapping tablet: real 0.016s user 0.006s sys 0.004s
I20260809 22:52:28.052080 17519 catalog_manager.cc:4841] TS 7cc5e8e5db8840fda0686859e4c61823 (127.15.113.130:37287): tablet eae23c2246fa417a82a5ef0656420633 (table test-table [id=7a9c4d4047e8401ab08a24c180b2bb2f]) successfully deleted
I20260809 22:52:28.052703 18320 raft_consensus.cc:348] T f61d736329a84372a0814ab03a66f100 P 610b38e390064672bc4c03a90c4bb702 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610b38e390064672bc4c03a90c4bb702" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 45313 } } peers { permanent_uuid: "7cc5e8e5db8840fda0686859e4c61823" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 37287 } } peers { permanent_uuid: "6d8a1e80214f4c0192bf6f3f3c7d5b19" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 36465 } }
I20260809 22:52:28.052877 18320 raft_consensus.cc:374] T f61d736329a84372a0814ab03a66f100 P 610b38e390064672bc4c03a90c4bb702 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:52:28.052983 18320 raft_consensus.cc:729] T f61d736329a84372a0814ab03a66f100 P 610b38e390064672bc4c03a90c4bb702 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 610b38e390064672bc4c03a90c4bb702, State: Initialized, Role: FOLLOWER
I20260809 22:52:28.052877 18318 raft_consensus.cc:348] T f61d736329a84372a0814ab03a66f100 P 7cc5e8e5db8840fda0686859e4c61823 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610b38e390064672bc4c03a90c4bb702" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 45313 } } peers { permanent_uuid: "7cc5e8e5db8840fda0686859e4c61823" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 37287 } } peers { permanent_uuid: "6d8a1e80214f4c0192bf6f3f3c7d5b19" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 36465 } }
I20260809 22:52:28.053180 18318 raft_consensus.cc:374] T f61d736329a84372a0814ab03a66f100 P 7cc5e8e5db8840fda0686859e4c61823 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:52:28.053293 18318 raft_consensus.cc:729] T f61d736329a84372a0814ab03a66f100 P 7cc5e8e5db8840fda0686859e4c61823 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7cc5e8e5db8840fda0686859e4c61823, State: Initialized, Role: FOLLOWER
I20260809 22:52:28.053236 18320 consensus_queue.cc:260] T f61d736329a84372a0814ab03a66f100 P 610b38e390064672bc4c03a90c4bb702 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610b38e390064672bc4c03a90c4bb702" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 45313 } } peers { permanent_uuid: "7cc5e8e5db8840fda0686859e4c61823" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 37287 } } peers { permanent_uuid: "6d8a1e80214f4c0192bf6f3f3c7d5b19" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 36465 } }
I20260809 22:52:28.053588 18318 consensus_queue.cc:260] T f61d736329a84372a0814ab03a66f100 P 7cc5e8e5db8840fda0686859e4c61823 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610b38e390064672bc4c03a90c4bb702" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 45313 } } peers { permanent_uuid: "7cc5e8e5db8840fda0686859e4c61823" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 37287 } } peers { permanent_uuid: "6d8a1e80214f4c0192bf6f3f3c7d5b19" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 36465 } }
I20260809 22:52:28.053833 18320 ts_tablet_manager.cc:1428] T f61d736329a84372a0814ab03a66f100 P 610b38e390064672bc4c03a90c4bb702: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260809 22:52:28.054354 18318 ts_tablet_manager.cc:1428] T f61d736329a84372a0814ab03a66f100 P 7cc5e8e5db8840fda0686859e4c61823: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
I20260809 22:52:28.057049 18321 tablet_bootstrap.cc:492] T f61d736329a84372a0814ab03a66f100 P 6d8a1e80214f4c0192bf6f3f3c7d5b19: No bootstrap required, opened a new log
I20260809 22:52:28.057222 18321 ts_tablet_manager.cc:1397] T f61d736329a84372a0814ab03a66f100 P 6d8a1e80214f4c0192bf6f3f3c7d5b19: Time spent bootstrapping tablet: real 0.025s user 0.013s sys 0.004s
I20260809 22:52:28.058131 18321 raft_consensus.cc:348] T f61d736329a84372a0814ab03a66f100 P 6d8a1e80214f4c0192bf6f3f3c7d5b19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610b38e390064672bc4c03a90c4bb702" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 45313 } } peers { permanent_uuid: "7cc5e8e5db8840fda0686859e4c61823" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 37287 } } peers { permanent_uuid: "6d8a1e80214f4c0192bf6f3f3c7d5b19" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 36465 } }
I20260809 22:52:28.058362 18321 raft_consensus.cc:374] T f61d736329a84372a0814ab03a66f100 P 6d8a1e80214f4c0192bf6f3f3c7d5b19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:52:28.058467 18321 raft_consensus.cc:729] T f61d736329a84372a0814ab03a66f100 P 6d8a1e80214f4c0192bf6f3f3c7d5b19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6d8a1e80214f4c0192bf6f3f3c7d5b19, State: Initialized, Role: FOLLOWER
I20260809 22:52:28.058790 18321 consensus_queue.cc:260] T f61d736329a84372a0814ab03a66f100 P 6d8a1e80214f4c0192bf6f3f3c7d5b19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610b38e390064672bc4c03a90c4bb702" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 45313 } } peers { permanent_uuid: "7cc5e8e5db8840fda0686859e4c61823" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 37287 } } peers { permanent_uuid: "6d8a1e80214f4c0192bf6f3f3c7d5b19" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 36465 } }
I20260809 22:52:28.061640 18321 ts_tablet_manager.cc:1428] T f61d736329a84372a0814ab03a66f100 P 6d8a1e80214f4c0192bf6f3f3c7d5b19: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260809 22:52:28.237015 18353 raft_consensus.cc:482] T f61d736329a84372a0814ab03a66f100 P 610b38e390064672bc4c03a90c4bb702 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 22:52:28.237274 18353 raft_consensus.cc:504] T f61d736329a84372a0814ab03a66f100 P 610b38e390064672bc4c03a90c4bb702 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610b38e390064672bc4c03a90c4bb702" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 45313 } } peers { permanent_uuid: "7cc5e8e5db8840fda0686859e4c61823" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 37287 } } peers { permanent_uuid: "6d8a1e80214f4c0192bf6f3f3c7d5b19" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 36465 } }
I20260809 22:52:28.238240 18353 leader_election.cc:288] T f61d736329a84372a0814ab03a66f100 P 610b38e390064672bc4c03a90c4bb702 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7cc5e8e5db8840fda0686859e4c61823 (127.15.113.130:37287), 6d8a1e80214f4c0192bf6f3f3c7d5b19 (127.15.113.131:36465)
I20260809 22:52:28.238832 17992 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f61d736329a84372a0814ab03a66f100" candidate_uuid: "610b38e390064672bc4c03a90c4bb702" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cc5e8e5db8840fda0686859e4c61823" is_pre_election: true
I20260809 22:52:28.238837 18212 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f61d736329a84372a0814ab03a66f100" candidate_uuid: "610b38e390064672bc4c03a90c4bb702" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6d8a1e80214f4c0192bf6f3f3c7d5b19" is_pre_election: true
I20260809 22:52:28.239077 17992 raft_consensus.cc:2454] T f61d736329a84372a0814ab03a66f100 P 7cc5e8e5db8840fda0686859e4c61823 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610b38e390064672bc4c03a90c4bb702 in term 0.
I20260809 22:52:28.239109 18212 raft_consensus.cc:2454] T f61d736329a84372a0814ab03a66f100 P 6d8a1e80214f4c0192bf6f3f3c7d5b19 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610b38e390064672bc4c03a90c4bb702 in term 0.
I20260809 22:52:28.239519 17673 leader_election.cc:302] T f61d736329a84372a0814ab03a66f100 P 610b38e390064672bc4c03a90c4bb702 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610b38e390064672bc4c03a90c4bb702, 7cc5e8e5db8840fda0686859e4c61823; no voters:
I20260809 22:52:28.239830 18353 raft_consensus.cc:2781] T f61d736329a84372a0814ab03a66f100 P 610b38e390064672bc4c03a90c4bb702 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 22:52:28.239950 18353 raft_consensus.cc:482] T f61d736329a84372a0814ab03a66f100 P 610b38e390064672bc4c03a90c4bb702 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 22:52:28.240038 18353 raft_consensus.cc:3037] T f61d736329a84372a0814ab03a66f100 P 610b38e390064672bc4c03a90c4bb702 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:52:28.242856 18353 raft_consensus.cc:504] T f61d736329a84372a0814ab03a66f100 P 610b38e390064672bc4c03a90c4bb702 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610b38e390064672bc4c03a90c4bb702" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 45313 } } peers { permanent_uuid: "7cc5e8e5db8840fda0686859e4c61823" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 37287 } } peers { permanent_uuid: "6d8a1e80214f4c0192bf6f3f3c7d5b19" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 36465 } }
I20260809 22:52:28.243655 18353 leader_election.cc:288] T f61d736329a84372a0814ab03a66f100 P 610b38e390064672bc4c03a90c4bb702 [CANDIDATE]: Term 1 election: Requested vote from peers 7cc5e8e5db8840fda0686859e4c61823 (127.15.113.130:37287), 6d8a1e80214f4c0192bf6f3f3c7d5b19 (127.15.113.131:36465)
I20260809 22:52:28.244217 18212 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f61d736329a84372a0814ab03a66f100" candidate_uuid: "610b38e390064672bc4c03a90c4bb702" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6d8a1e80214f4c0192bf6f3f3c7d5b19"
I20260809 22:52:28.244410 18212 raft_consensus.cc:3037] T f61d736329a84372a0814ab03a66f100 P 6d8a1e80214f4c0192bf6f3f3c7d5b19 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:52:28.244354 17992 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f61d736329a84372a0814ab03a66f100" candidate_uuid: "610b38e390064672bc4c03a90c4bb702" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cc5e8e5db8840fda0686859e4c61823"
I20260809 22:52:28.244554 17992 raft_consensus.cc:3037] T f61d736329a84372a0814ab03a66f100 P 7cc5e8e5db8840fda0686859e4c61823 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:52:28.246520 17992 raft_consensus.cc:2454] T f61d736329a84372a0814ab03a66f100 P 7cc5e8e5db8840fda0686859e4c61823 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610b38e390064672bc4c03a90c4bb702 in term 1.
I20260809 22:52:28.246888 17673 leader_election.cc:302] T f61d736329a84372a0814ab03a66f100 P 610b38e390064672bc4c03a90c4bb702 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610b38e390064672bc4c03a90c4bb702, 7cc5e8e5db8840fda0686859e4c61823; no voters:
I20260809 22:52:28.247152 18353 raft_consensus.cc:2781] T f61d736329a84372a0814ab03a66f100 P 610b38e390064672bc4c03a90c4bb702 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 22:52:28.247223 18212 raft_consensus.cc:2454] T f61d736329a84372a0814ab03a66f100 P 6d8a1e80214f4c0192bf6f3f3c7d5b19 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610b38e390064672bc4c03a90c4bb702 in term 1.
I20260809 22:52:28.247308 18353 raft_consensus.cc:686] T f61d736329a84372a0814ab03a66f100 P 610b38e390064672bc4c03a90c4bb702 [term 1 LEADER]: Becoming Leader. State: Replica: 610b38e390064672bc4c03a90c4bb702, State: Running, Role: LEADER
I20260809 22:52:28.247597 18353 consensus_queue.cc:237] T f61d736329a84372a0814ab03a66f100 P 610b38e390064672bc4c03a90c4bb702 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610b38e390064672bc4c03a90c4bb702" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 45313 } } peers { permanent_uuid: "7cc5e8e5db8840fda0686859e4c61823" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 37287 } } peers { permanent_uuid: "6d8a1e80214f4c0192bf6f3f3c7d5b19" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 36465 } }
I20260809 22:52:28.250444 17540 catalog_manager.cc:5495] T f61d736329a84372a0814ab03a66f100 P 610b38e390064672bc4c03a90c4bb702 reported cstate change: term changed from 0 to 1, leader changed from <none> to 610b38e390064672bc4c03a90c4bb702 (127.15.113.129). New cstate: current_term: 1 leader_uuid: "610b38e390064672bc4c03a90c4bb702" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610b38e390064672bc4c03a90c4bb702" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 45313 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7cc5e8e5db8840fda0686859e4c61823" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 37287 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6d8a1e80214f4c0192bf6f3f3c7d5b19" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 36465 } health_report { overall_health: UNKNOWN } } }
Aug 09 22:52:28 dist-test-slave-ncp9 krb5kdc[17476](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315945, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.113.190@KRBTEST.COM for kudu/127.15.113.190@KRBTEST.COM
I20260809 22:52:28.308821 18381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:52:28.295088 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42394 (local address 127.15.113.190:38741)
0809 22:52:28.295906 (+ 818us) server_negotiation.cc:206] Beginning negotiation
0809 22:52:28.295911 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:52:28.295930 (+ 19us) server_negotiation.cc:407] Connection header received
0809 22:52:28.295969 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:52:28.295972 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:52:28.296030 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:52:28.296127 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:52:28.297226 (+ 1099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:52:28.297982 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:52:28.299168 (+ 1186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:52:28.299559 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:52:28.299663 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:52:28.303242 (+ 3579us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:52:28.303247 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:52:28.303250 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:52:28.303279 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:52:28.305831 (+ 2552us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:52:28.306673 (+ 842us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:52:28.306677 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:52:28.306679 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:52:28.306789 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:52:28.307272 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:52:28.307275 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:52:28.307277 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:52:28.307559 (+ 282us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:52:28.307666 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0809 22:52:28.308215 (+ 549us) server_negotiation.cc:299] Negotiation successful
0809 22:52:28.308516 (+ 301us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":121,"server-negotiator.queue_time_us":675,"thread_start_us":547,"threads_started":1}
I20260809 22:52:28.308862 18380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:52:28.294995 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:38741 (local address 127.0.0.1:42394)
0809 22:52:28.295598 (+ 603us) negotiation.cc:107] Waiting for socket to connect
0809 22:52:28.295635 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0809 22:52:28.295803 (+ 168us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:52:28.296165 (+ 362us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:52:28.296169 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:52:28.296815 (+ 646us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:52:28.297061 (+ 246us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:52:28.297071 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:52:28.298101 (+ 1030us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:52:28.298105 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:52:28.299032 (+ 927us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:52:28.299044 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:52:28.299710 (+ 666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:52:28.299715 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:52:28.299866 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:52:28.300541 (+ 675us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:52:28.300558 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:52:28.303037 (+ 2479us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:52:28.305976 (+ 2939us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:52:28.305989 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:52:28.305997 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:52:28.306567 (+ 570us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:52:28.306894 (+ 327us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:52:28.306900 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:52:28.306904 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:52:28.307167 (+ 263us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:52:28.307693 (+ 526us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:52:28.307702 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:52:28.307897 (+ 195us) client_negotiation.cc:769] Sending connection context
0809 22:52:28.308102 (+ 205us) client_negotiation.cc:240] Negotiation successful
0809 22:52:28.308399 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":447,"thread_start_us":305,"threads_started":1}
I20260809 22:52:28.314075 17540 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.15.113.190@KRBTEST.COM'} at 127.0.0.1:42394:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260809 22:52:28.315032 17540 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 22:52:28.321381 17751 tablet_service.cc:1467] Processing CreateTablet for tablet 54b4b78c839940208f632cead25a10fa (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=354244ef1fcd4a7aa7d1d68016a3b60b]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 22:52:28.321458 17963 tablet_service.cc:1467] Processing CreateTablet for tablet 54b4b78c839940208f632cead25a10fa (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=354244ef1fcd4a7aa7d1d68016a3b60b]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 22:52:28.321911 17751 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 54b4b78c839940208f632cead25a10fa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:52:28.322029 17963 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 54b4b78c839940208f632cead25a10fa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:52:28.327111 18320 tablet_bootstrap.cc:492] T 54b4b78c839940208f632cead25a10fa P 610b38e390064672bc4c03a90c4bb702: Bootstrap starting.
I20260809 22:52:28.327638 18178 tablet_service.cc:1467] Processing CreateTablet for tablet 54b4b78c839940208f632cead25a10fa (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=354244ef1fcd4a7aa7d1d68016a3b60b]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 22:52:28.328197 18178 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 54b4b78c839940208f632cead25a10fa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:52:28.329007 18320 tablet_bootstrap.cc:654] T 54b4b78c839940208f632cead25a10fa P 610b38e390064672bc4c03a90c4bb702: Neither blocks nor log segments found. Creating new log.
I20260809 22:52:28.329248 18318 tablet_bootstrap.cc:492] T 54b4b78c839940208f632cead25a10fa P 7cc5e8e5db8840fda0686859e4c61823: Bootstrap starting.
I20260809 22:52:28.330910 18318 tablet_bootstrap.cc:654] T 54b4b78c839940208f632cead25a10fa P 7cc5e8e5db8840fda0686859e4c61823: Neither blocks nor log segments found. Creating new log.
I20260809 22:52:28.336794 18321 tablet_bootstrap.cc:492] T 54b4b78c839940208f632cead25a10fa P 6d8a1e80214f4c0192bf6f3f3c7d5b19: Bootstrap starting.
I20260809 22:52:28.338222 18320 tablet_bootstrap.cc:492] T 54b4b78c839940208f632cead25a10fa P 610b38e390064672bc4c03a90c4bb702: No bootstrap required, opened a new log
I20260809 22:52:28.338218 18318 tablet_bootstrap.cc:492] T 54b4b78c839940208f632cead25a10fa P 7cc5e8e5db8840fda0686859e4c61823: No bootstrap required, opened a new log
I20260809 22:52:28.338397 18320 ts_tablet_manager.cc:1397] T 54b4b78c839940208f632cead25a10fa P 610b38e390064672bc4c03a90c4bb702: Time spent bootstrapping tablet: real 0.011s user 0.004s sys 0.006s
I20260809 22:52:28.338445 18318 ts_tablet_manager.cc:1397] T 54b4b78c839940208f632cead25a10fa P 7cc5e8e5db8840fda0686859e4c61823: Time spent bootstrapping tablet: real 0.009s user 0.000s sys 0.006s
I20260809 22:52:28.339183 18320 raft_consensus.cc:348] T 54b4b78c839940208f632cead25a10fa P 610b38e390064672bc4c03a90c4bb702 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610b38e390064672bc4c03a90c4bb702" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 45313 } } peers { permanent_uuid: "7cc5e8e5db8840fda0686859e4c61823" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 37287 } } peers { permanent_uuid: "6d8a1e80214f4c0192bf6f3f3c7d5b19" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 36465 } }
I20260809 22:52:28.339191 18318 raft_consensus.cc:348] T 54b4b78c839940208f632cead25a10fa P 7cc5e8e5db8840fda0686859e4c61823 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610b38e390064672bc4c03a90c4bb702" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 45313 } } peers { permanent_uuid: "7cc5e8e5db8840fda0686859e4c61823" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 37287 } } peers { permanent_uuid: "6d8a1e80214f4c0192bf6f3f3c7d5b19" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 36465 } }
I20260809 22:52:28.339440 18320 raft_consensus.cc:374] T 54b4b78c839940208f632cead25a10fa P 610b38e390064672bc4c03a90c4bb702 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:52:28.339527 18320 raft_consensus.cc:729] T 54b4b78c839940208f632cead25a10fa P 610b38e390064672bc4c03a90c4bb702 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 610b38e390064672bc4c03a90c4bb702, State: Initialized, Role: FOLLOWER
I20260809 22:52:28.338581 18321 tablet_bootstrap.cc:654] T 54b4b78c839940208f632cead25a10fa P 6d8a1e80214f4c0192bf6f3f3c7d5b19: Neither blocks nor log segments found. Creating new log.
I20260809 22:52:28.339555 18318 raft_consensus.cc:374] T 54b4b78c839940208f632cead25a10fa P 7cc5e8e5db8840fda0686859e4c61823 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:52:28.339740 18318 raft_consensus.cc:729] T 54b4b78c839940208f632cead25a10fa P 7cc5e8e5db8840fda0686859e4c61823 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7cc5e8e5db8840fda0686859e4c61823, State: Initialized, Role: FOLLOWER
I20260809 22:52:28.339838 18320 consensus_queue.cc:260] T 54b4b78c839940208f632cead25a10fa P 610b38e390064672bc4c03a90c4bb702 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610b38e390064672bc4c03a90c4bb702" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 45313 } } peers { permanent_uuid: "7cc5e8e5db8840fda0686859e4c61823" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 37287 } } peers { permanent_uuid: "6d8a1e80214f4c0192bf6f3f3c7d5b19" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 36465 } }
I20260809 22:52:28.340266 18353 tablet_replica.cc:439] T 54b4b78c839940208f632cead25a10fa P 610b38e390064672bc4c03a90c4bb702: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610b38e390064672bc4c03a90c4bb702" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 45313 } } peers { permanent_uuid: "7cc5e8e5db8840fda0686859e4c61823" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 37287 } } peers { permanent_uuid: "6d8a1e80214f4c0192bf6f3f3c7d5b19" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 36465 } } }
I20260809 22:52:28.340548 18353 tablet_replica.cc:442] T 54b4b78c839940208f632cead25a10fa P 610b38e390064672bc4c03a90c4bb702: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 22:52:28.340445 18318 consensus_queue.cc:260] T 54b4b78c839940208f632cead25a10fa P 7cc5e8e5db8840fda0686859e4c61823 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610b38e390064672bc4c03a90c4bb702" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 45313 } } peers { permanent_uuid: "7cc5e8e5db8840fda0686859e4c61823" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 37287 } } peers { permanent_uuid: "6d8a1e80214f4c0192bf6f3f3c7d5b19" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 36465 } }
I20260809 22:52:28.341109 18328 tablet_replica.cc:439] T 54b4b78c839940208f632cead25a10fa P 7cc5e8e5db8840fda0686859e4c61823: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610b38e390064672bc4c03a90c4bb702" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 45313 } } peers { permanent_uuid: "7cc5e8e5db8840fda0686859e4c61823" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 37287 } } peers { permanent_uuid: "6d8a1e80214f4c0192bf6f3f3c7d5b19" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 36465 } } }
I20260809 22:52:28.341472 18328 tablet_replica.cc:442] T 54b4b78c839940208f632cead25a10fa P 7cc5e8e5db8840fda0686859e4c61823: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 22:52:28.342008 18320 ts_tablet_manager.cc:1428] T 54b4b78c839940208f632cead25a10fa P 610b38e390064672bc4c03a90c4bb702: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20260809 22:52:28.342449 18318 ts_tablet_manager.cc:1428] T 54b4b78c839940208f632cead25a10fa P 7cc5e8e5db8840fda0686859e4c61823: Time spent starting tablet: real 0.004s user 0.000s sys 0.001s
I20260809 22:52:28.344503 18321 tablet_bootstrap.cc:492] T 54b4b78c839940208f632cead25a10fa P 6d8a1e80214f4c0192bf6f3f3c7d5b19: No bootstrap required, opened a new log
I20260809 22:52:28.344735 18321 ts_tablet_manager.cc:1397] T 54b4b78c839940208f632cead25a10fa P 6d8a1e80214f4c0192bf6f3f3c7d5b19: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.003s
I20260809 22:52:28.345541 18321 raft_consensus.cc:348] T 54b4b78c839940208f632cead25a10fa P 6d8a1e80214f4c0192bf6f3f3c7d5b19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610b38e390064672bc4c03a90c4bb702" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 45313 } } peers { permanent_uuid: "7cc5e8e5db8840fda0686859e4c61823" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 37287 } } peers { permanent_uuid: "6d8a1e80214f4c0192bf6f3f3c7d5b19" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 36465 } }
I20260809 22:52:28.345853 18321 raft_consensus.cc:374] T 54b4b78c839940208f632cead25a10fa P 6d8a1e80214f4c0192bf6f3f3c7d5b19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:52:28.345985 18321 raft_consensus.cc:729] T 54b4b78c839940208f632cead25a10fa P 6d8a1e80214f4c0192bf6f3f3c7d5b19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6d8a1e80214f4c0192bf6f3f3c7d5b19, State: Initialized, Role: FOLLOWER
I20260809 22:52:28.346315 18321 consensus_queue.cc:260] T 54b4b78c839940208f632cead25a10fa P 6d8a1e80214f4c0192bf6f3f3c7d5b19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610b38e390064672bc4c03a90c4bb702" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 45313 } } peers { permanent_uuid: "7cc5e8e5db8840fda0686859e4c61823" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 37287 } } peers { permanent_uuid: "6d8a1e80214f4c0192bf6f3f3c7d5b19" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 36465 } }
I20260809 22:52:28.346860 18321 ts_tablet_manager.cc:1428] T 54b4b78c839940208f632cead25a10fa P 6d8a1e80214f4c0192bf6f3f3c7d5b19: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260809 22:52:28.346805 18329 tablet_replica.cc:439] T 54b4b78c839940208f632cead25a10fa P 6d8a1e80214f4c0192bf6f3f3c7d5b19: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610b38e390064672bc4c03a90c4bb702" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 45313 } } peers { permanent_uuid: "7cc5e8e5db8840fda0686859e4c61823" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 37287 } } peers { permanent_uuid: "6d8a1e80214f4c0192bf6f3f3c7d5b19" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 36465 } } }
I20260809 22:52:28.347147 18329 tablet_replica.cc:442] T 54b4b78c839940208f632cead25a10fa P 6d8a1e80214f4c0192bf6f3f3c7d5b19: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 22:52:28.561908 18329 raft_consensus.cc:482] T 54b4b78c839940208f632cead25a10fa P 6d8a1e80214f4c0192bf6f3f3c7d5b19 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 22:52:28.562103 18329 raft_consensus.cc:504] T 54b4b78c839940208f632cead25a10fa P 6d8a1e80214f4c0192bf6f3f3c7d5b19 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610b38e390064672bc4c03a90c4bb702" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 45313 } } peers { permanent_uuid: "7cc5e8e5db8840fda0686859e4c61823" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 37287 } } peers { permanent_uuid: "6d8a1e80214f4c0192bf6f3f3c7d5b19" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 36465 } }
I20260809 22:52:28.563488 18329 leader_election.cc:288] T 54b4b78c839940208f632cead25a10fa P 6d8a1e80214f4c0192bf6f3f3c7d5b19 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 610b38e390064672bc4c03a90c4bb702 (127.15.113.129:45313), 7cc5e8e5db8840fda0686859e4c61823 (127.15.113.130:37287)
I20260809 22:52:28.570298 18391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:52:28.563820 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.113.131:39011 (local address 127.15.113.129:45313)
0809 22:52:28.564278 (+ 458us) server_negotiation.cc:206] Beginning negotiation
0809 22:52:28.564285 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:52:28.564309 (+ 24us) server_negotiation.cc:407] Connection header received
0809 22:52:28.564542 (+ 233us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:52:28.564548 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:52:28.564640 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:52:28.564741 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:52:28.565351 (+ 610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:52:28.566347 (+ 996us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:52:28.568605 (+ 2258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:52:28.569420 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:52:28.569567 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:52:28.569645 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0809 22:52:28.569899 (+ 254us) server_negotiation.cc:299] Negotiation successful
0809 22:52:28.570034 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":332,"thread_start_us":235,"threads_started":1}
I20260809 22:52:28.570313 18390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:52:28.563723 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.129:45313 (local address 127.15.113.131:39011)
0809 22:52:28.564217 (+ 494us) negotiation.cc:107] Waiting for socket to connect
0809 22:52:28.564233 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 22:52:28.564388 (+ 155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:52:28.564964 (+ 576us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:52:28.564969 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:52:28.564997 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:52:28.565197 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:52:28.565206 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:52:28.566468 (+ 1262us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:52:28.566472 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:52:28.568452 (+ 1980us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:52:28.568467 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:52:28.569569 (+ 1102us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:52:28.569575 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:52:28.569748 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:52:28.569772 (+ 24us) client_negotiation.cc:769] Sending connection context
0809 22:52:28.569859 (+ 87us) client_negotiation.cc:240] Negotiation successful
0809 22:52:28.569994 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":334,"thread_start_us":223,"threads_started":1}
I20260809 22:52:28.570834 18392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:52:28.564350 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.130:37287 (local address 127.15.113.131:58565)
0809 22:52:28.564819 (+ 469us) negotiation.cc:107] Waiting for socket to connect
0809 22:52:28.564836 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 22:52:28.565019 (+ 183us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:52:28.565311 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:52:28.565315 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:52:28.565340 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:52:28.565549 (+ 209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:52:28.565556 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:52:28.566818 (+ 1262us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:52:28.566823 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:52:28.568762 (+ 1939us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:52:28.568773 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:52:28.569853 (+ 1080us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:52:28.569859 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:52:28.570019 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:52:28.570043 (+ 24us) client_negotiation.cc:769] Sending connection context
0809 22:52:28.570130 (+ 87us) client_negotiation.cc:240] Negotiation successful
0809 22:52:28.570257 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":361,"mutex_wait_us":1,"thread_start_us":218,"threads_started":1}
I20260809 22:52:28.570978 18393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:52:28.564497 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.113.131:58565 (local address 127.15.113.130:37287)
0809 22:52:28.565034 (+ 537us) server_negotiation.cc:206] Beginning negotiation
0809 22:52:28.565038 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 22:52:28.565060 (+ 22us) server_negotiation.cc:407] Connection header received
0809 22:52:28.565121 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:52:28.565124 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:52:28.565194 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:52:28.565273 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:52:28.565688 (+ 415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:52:28.566689 (+ 1001us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:52:28.568935 (+ 2246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:52:28.569678 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:52:28.569828 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:52:28.569898 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0809 22:52:28.570163 (+ 265us) server_negotiation.cc:299] Negotiation successful
0809 22:52:28.570306 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":440,"thread_start_us":301,"threads_started":1}
I20260809 22:52:28.571234 17786 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "54b4b78c839940208f632cead25a10fa" candidate_uuid: "6d8a1e80214f4c0192bf6f3f3c7d5b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "610b38e390064672bc4c03a90c4bb702" is_pre_election: true
I20260809 22:52:28.571547 17992 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "54b4b78c839940208f632cead25a10fa" candidate_uuid: "6d8a1e80214f4c0192bf6f3f3c7d5b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cc5e8e5db8840fda0686859e4c61823" is_pre_election: true
I20260809 22:52:28.571625 17786 raft_consensus.cc:2454] T 54b4b78c839940208f632cead25a10fa P 610b38e390064672bc4c03a90c4bb702 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6d8a1e80214f4c0192bf6f3f3c7d5b19 in term 0.
I20260809 22:52:28.571785 17992 raft_consensus.cc:2454] T 54b4b78c839940208f632cead25a10fa P 7cc5e8e5db8840fda0686859e4c61823 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6d8a1e80214f4c0192bf6f3f3c7d5b19 in term 0.
I20260809 22:52:28.572165 18107 leader_election.cc:302] T 54b4b78c839940208f632cead25a10fa P 6d8a1e80214f4c0192bf6f3f3c7d5b19 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610b38e390064672bc4c03a90c4bb702, 6d8a1e80214f4c0192bf6f3f3c7d5b19; no voters:
I20260809 22:52:28.572503 18329 raft_consensus.cc:2781] T 54b4b78c839940208f632cead25a10fa P 6d8a1e80214f4c0192bf6f3f3c7d5b19 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 22:52:28.572621 18329 raft_consensus.cc:482] T 54b4b78c839940208f632cead25a10fa P 6d8a1e80214f4c0192bf6f3f3c7d5b19 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 22:52:28.572719 18329 raft_consensus.cc:3037] T 54b4b78c839940208f632cead25a10fa P 6d8a1e80214f4c0192bf6f3f3c7d5b19 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:52:28.574834 18329 raft_consensus.cc:504] T 54b4b78c839940208f632cead25a10fa P 6d8a1e80214f4c0192bf6f3f3c7d5b19 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610b38e390064672bc4c03a90c4bb702" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 45313 } } peers { permanent_uuid: "7cc5e8e5db8840fda0686859e4c61823" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 37287 } } peers { permanent_uuid: "6d8a1e80214f4c0192bf6f3f3c7d5b19" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 36465 } }
I20260809 22:52:28.575327 18329 leader_election.cc:288] T 54b4b78c839940208f632cead25a10fa P 6d8a1e80214f4c0192bf6f3f3c7d5b19 [CANDIDATE]: Term 1 election: Requested vote from peers 610b38e390064672bc4c03a90c4bb702 (127.15.113.129:45313), 7cc5e8e5db8840fda0686859e4c61823 (127.15.113.130:37287)
I20260809 22:52:28.575735 17786 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "54b4b78c839940208f632cead25a10fa" candidate_uuid: "6d8a1e80214f4c0192bf6f3f3c7d5b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "610b38e390064672bc4c03a90c4bb702"
I20260809 22:52:28.575791 17992 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "54b4b78c839940208f632cead25a10fa" candidate_uuid: "6d8a1e80214f4c0192bf6f3f3c7d5b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cc5e8e5db8840fda0686859e4c61823"
I20260809 22:52:28.575902 17786 raft_consensus.cc:3037] T 54b4b78c839940208f632cead25a10fa P 610b38e390064672bc4c03a90c4bb702 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:52:28.575981 17992 raft_consensus.cc:3037] T 54b4b78c839940208f632cead25a10fa P 7cc5e8e5db8840fda0686859e4c61823 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:52:28.577921 17992 raft_consensus.cc:2454] T 54b4b78c839940208f632cead25a10fa P 7cc5e8e5db8840fda0686859e4c61823 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6d8a1e80214f4c0192bf6f3f3c7d5b19 in term 1.
I20260809 22:52:28.578001 17786 raft_consensus.cc:2454] T 54b4b78c839940208f632cead25a10fa P 610b38e390064672bc4c03a90c4bb702 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6d8a1e80214f4c0192bf6f3f3c7d5b19 in term 1.
I20260809 22:52:28.578262 18107 leader_election.cc:302] T 54b4b78c839940208f632cead25a10fa P 6d8a1e80214f4c0192bf6f3f3c7d5b19 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6d8a1e80214f4c0192bf6f3f3c7d5b19, 7cc5e8e5db8840fda0686859e4c61823; no voters:
I20260809 22:52:28.578526 18329 raft_consensus.cc:2781] T 54b4b78c839940208f632cead25a10fa P 6d8a1e80214f4c0192bf6f3f3c7d5b19 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 22:52:28.578954 18329 raft_consensus.cc:686] T 54b4b78c839940208f632cead25a10fa P 6d8a1e80214f4c0192bf6f3f3c7d5b19 [term 1 LEADER]: Becoming Leader. State: Replica: 6d8a1e80214f4c0192bf6f3f3c7d5b19, State: Running, Role: LEADER
I20260809 22:52:28.579288 18329 consensus_queue.cc:237] T 54b4b78c839940208f632cead25a10fa P 6d8a1e80214f4c0192bf6f3f3c7d5b19 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610b38e390064672bc4c03a90c4bb702" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 45313 } } peers { permanent_uuid: "7cc5e8e5db8840fda0686859e4c61823" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 37287 } } peers { permanent_uuid: "6d8a1e80214f4c0192bf6f3f3c7d5b19" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 36465 } }
I20260809 22:52:28.581169 18395 tablet_replica.cc:439] T 54b4b78c839940208f632cead25a10fa P 6d8a1e80214f4c0192bf6f3f3c7d5b19: TxnStatusTablet state changed. Reason: New leader 6d8a1e80214f4c0192bf6f3f3c7d5b19. Latest consensus state: current_term: 1 leader_uuid: "6d8a1e80214f4c0192bf6f3f3c7d5b19" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610b38e390064672bc4c03a90c4bb702" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 45313 } } peers { permanent_uuid: "7cc5e8e5db8840fda0686859e4c61823" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 37287 } } peers { permanent_uuid: "6d8a1e80214f4c0192bf6f3f3c7d5b19" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 36465 } } }
I20260809 22:52:28.581506 18395 tablet_replica.cc:442] T 54b4b78c839940208f632cead25a10fa P 6d8a1e80214f4c0192bf6f3f3c7d5b19: This TxnStatusTablet replica's current role is: LEADER
I20260809 22:52:28.582144 18398 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260809 22:52:28.582391 18398 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260809 22:52:28.582841 17532 catalog_manager.cc:5495] T 54b4b78c839940208f632cead25a10fa P 6d8a1e80214f4c0192bf6f3f3c7d5b19 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6d8a1e80214f4c0192bf6f3f3c7d5b19 (127.15.113.131). New cstate: current_term: 1 leader_uuid: "6d8a1e80214f4c0192bf6f3f3c7d5b19" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610b38e390064672bc4c03a90c4bb702" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 45313 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7cc5e8e5db8840fda0686859e4c61823" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 37287 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6d8a1e80214f4c0192bf6f3f3c7d5b19" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 36465 } health_report { overall_health: HEALTHY } } }
I20260809 22:52:28.641076 18353 consensus_queue.cc:1035] T f61d736329a84372a0814ab03a66f100 P 610b38e390064672bc4c03a90c4bb702 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6d8a1e80214f4c0192bf6f3f3c7d5b19" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 36465 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 22:52:28.663049 18339 consensus_queue.cc:1035] T f61d736329a84372a0814ab03a66f100 P 610b38e390064672bc4c03a90c4bb702 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cc5e8e5db8840fda0686859e4c61823" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 37287 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 22:52:28.675357 18380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:52:28.646758 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.131:36465 (local address 127.0.0.1:56812)
0809 22:52:28.647569 (+ 811us) negotiation.cc:107] Waiting for socket to connect
0809 22:52:28.647591 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0809 22:52:28.647727 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:52:28.662727 (+ 15000us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:52:28.662732 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:52:28.662747 (+ 15us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 22:52:28.663049 (+ 302us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:52:28.663067 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:52:28.668708 (+ 5641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:52:28.668714 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:52:28.670790 (+ 2076us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:52:28.670808 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:52:28.672468 (+ 1660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:52:28.672475 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:52:28.672969 (+ 494us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:52:28.673062 (+ 93us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 22:52:28.674650 (+ 1588us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 22:52:28.674674 (+ 24us) client_negotiation.cc:769] Sending connection context
0809 22:52:28.674747 (+ 73us) client_negotiation.cc:240] Negotiation successful
0809 22:52:28.674944 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":589}
I20260809 22:52:28.678447 18402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:52:28.647229 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56812 (local address 127.15.113.131:36465)
0809 22:52:28.662085 (+ 14856us) server_negotiation.cc:206] Beginning negotiation
0809 22:52:28.662091 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:52:28.662148 (+ 57us) server_negotiation.cc:407] Connection header received
0809 22:52:28.662261 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:52:28.662266 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:52:28.662410 (+ 144us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:52:28.662590 (+ 180us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 22:52:28.664571 (+ 1981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:52:28.668536 (+ 3965us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:52:28.670966 (+ 2430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:52:28.672141 (+ 1175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:52:28.672262 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:52:28.673419 (+ 1157us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 22:52:28.674419 (+ 1000us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 22:52:28.674592 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0809 22:52:28.675851 (+ 1259us) server_negotiation.cc:299] Negotiation successful
0809 22:52:28.675936 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14196,"thread_start_us":8541,"threads_started":1}
I20260809 22:52:28.683687 17992 raft_consensus.cc:1261] T 54b4b78c839940208f632cead25a10fa P 7cc5e8e5db8840fda0686859e4c61823 [term 1 FOLLOWER]: Refusing update from remote peer 6d8a1e80214f4c0192bf6f3f3c7d5b19: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 22:52:28.683897 17786 raft_consensus.cc:1261] T 54b4b78c839940208f632cead25a10fa P 610b38e390064672bc4c03a90c4bb702 [term 1 FOLLOWER]: Refusing update from remote peer 6d8a1e80214f4c0192bf6f3f3c7d5b19: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 22:52:28.684486 18395 consensus_queue.cc:1035] T 54b4b78c839940208f632cead25a10fa P 6d8a1e80214f4c0192bf6f3f3c7d5b19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cc5e8e5db8840fda0686859e4c61823" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 37287 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 22:52:28.685400 18329 consensus_queue.cc:1035] T 54b4b78c839940208f632cead25a10fa P 6d8a1e80214f4c0192bf6f3f3c7d5b19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610b38e390064672bc4c03a90c4bb702" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 45313 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 22:52:28.700235 18405 tablet_replica.cc:439] T 54b4b78c839940208f632cead25a10fa P 610b38e390064672bc4c03a90c4bb702: TxnStatusTablet state changed. Reason: New leader 6d8a1e80214f4c0192bf6f3f3c7d5b19. Latest consensus state: current_term: 1 leader_uuid: "6d8a1e80214f4c0192bf6f3f3c7d5b19" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610b38e390064672bc4c03a90c4bb702" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 45313 } } peers { permanent_uuid: "7cc5e8e5db8840fda0686859e4c61823" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 37287 } } peers { permanent_uuid: "6d8a1e80214f4c0192bf6f3f3c7d5b19" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 36465 } } }
I20260809 22:52:28.700626 18405 tablet_replica.cc:442] T 54b4b78c839940208f632cead25a10fa P 610b38e390064672bc4c03a90c4bb702: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 22:52:28.702791 18328 tablet_replica.cc:439] T 54b4b78c839940208f632cead25a10fa P 7cc5e8e5db8840fda0686859e4c61823: TxnStatusTablet state changed. Reason: New leader 6d8a1e80214f4c0192bf6f3f3c7d5b19. Latest consensus state: current_term: 1 leader_uuid: "6d8a1e80214f4c0192bf6f3f3c7d5b19" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610b38e390064672bc4c03a90c4bb702" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 45313 } } peers { permanent_uuid: "7cc5e8e5db8840fda0686859e4c61823" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 37287 } } peers { permanent_uuid: "6d8a1e80214f4c0192bf6f3f3c7d5b19" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 36465 } } }
I20260809 22:52:28.703382 18328 tablet_replica.cc:442] T 54b4b78c839940208f632cead25a10fa P 7cc5e8e5db8840fda0686859e4c61823: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 22:52:28.711620 18411 mvcc.cc:205] Tried to move back new op lower bound from 7316750125799833600 to 7316750125384077312. Current Snapshot: MvccSnapshot[applied={T|T < 7316750125799833600}]
I20260809 22:52:28.708236 18329 tablet_replica.cc:439] T 54b4b78c839940208f632cead25a10fa P 6d8a1e80214f4c0192bf6f3f3c7d5b19: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "6d8a1e80214f4c0192bf6f3f3c7d5b19" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610b38e390064672bc4c03a90c4bb702" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 45313 } } peers { permanent_uuid: "7cc5e8e5db8840fda0686859e4c61823" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 37287 } } peers { permanent_uuid: "6d8a1e80214f4c0192bf6f3f3c7d5b19" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 36465 } } }
I20260809 22:52:28.719745 18329 tablet_replica.cc:442] T 54b4b78c839940208f632cead25a10fa P 6d8a1e80214f4c0192bf6f3f3c7d5b19: This TxnStatusTablet replica's current role is: LEADER
I20260809 22:52:28.723030 18329 tablet_replica.cc:439] T 54b4b78c839940208f632cead25a10fa P 6d8a1e80214f4c0192bf6f3f3c7d5b19: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "6d8a1e80214f4c0192bf6f3f3c7d5b19" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610b38e390064672bc4c03a90c4bb702" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 45313 } } peers { permanent_uuid: "7cc5e8e5db8840fda0686859e4c61823" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 37287 } } peers { permanent_uuid: "6d8a1e80214f4c0192bf6f3f3c7d5b19" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 36465 } } }
I20260809 22:52:28.723408 18329 tablet_replica.cc:442] T 54b4b78c839940208f632cead25a10fa P 6d8a1e80214f4c0192bf6f3f3c7d5b19: This TxnStatusTablet replica's current role is: LEADER
I20260809 22:52:28.733350 18405 tablet_replica.cc:439] T 54b4b78c839940208f632cead25a10fa P 610b38e390064672bc4c03a90c4bb702: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "6d8a1e80214f4c0192bf6f3f3c7d5b19" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610b38e390064672bc4c03a90c4bb702" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 45313 } } peers { permanent_uuid: "7cc5e8e5db8840fda0686859e4c61823" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 37287 } } peers { permanent_uuid: "6d8a1e80214f4c0192bf6f3f3c7d5b19" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 36465 } } }
I20260809 22:52:28.733763 18405 tablet_replica.cc:442] T 54b4b78c839940208f632cead25a10fa P 610b38e390064672bc4c03a90c4bb702: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 22:52:28.735380 18328 tablet_replica.cc:439] T 54b4b78c839940208f632cead25a10fa P 7cc5e8e5db8840fda0686859e4c61823: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "6d8a1e80214f4c0192bf6f3f3c7d5b19" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610b38e390064672bc4c03a90c4bb702" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 45313 } } peers { permanent_uuid: "7cc5e8e5db8840fda0686859e4c61823" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 37287 } } peers { permanent_uuid: "6d8a1e80214f4c0192bf6f3f3c7d5b19" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 36465 } } }
I20260809 22:52:28.735849 18328 tablet_replica.cc:442] T 54b4b78c839940208f632cead25a10fa P 7cc5e8e5db8840fda0686859e4c61823: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 22:52:28.749706 18402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:52:28.741815 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.113.129:57301 (local address 127.15.113.131:36465)
0809 22:52:28.742006 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0809 22:52:28.742012 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:52:28.742952 (+ 940us) server_negotiation.cc:407] Connection header received
0809 22:52:28.743116 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:52:28.743121 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:52:28.743217 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:52:28.743315 (+ 98us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 22:52:28.744206 (+ 891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:52:28.745675 (+ 1469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:52:28.747466 (+ 1791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:52:28.748280 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:52:28.748401 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:52:28.748914 (+ 513us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 22:52:28.749049 (+ 135us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 22:52:28.749145 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0809 22:52:28.749245 (+ 100us) server_negotiation.cc:299] Negotiation successful
0809 22:52:28.749325 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260809 22:52:28.749871 18421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:52:28.741894 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.131:36465 (local address 127.15.113.129:57301)
0809 22:52:28.742827 (+ 933us) negotiation.cc:107] Waiting for socket to connect
0809 22:52:28.742856 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0809 22:52:28.743001 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:52:28.743368 (+ 367us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:52:28.743372 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:52:28.743383 (+ 11us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 22:52:28.744010 (+ 627us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:52:28.744023 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:52:28.745858 (+ 1835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:52:28.745863 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:52:28.747323 (+ 1460us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:52:28.747336 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:52:28.748446 (+ 1110us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:52:28.748451 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:52:28.748625 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:52:28.748683 (+ 58us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 22:52:28.749136 (+ 453us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 22:52:28.749145 (+ 9us) client_negotiation.cc:769] Sending connection context
0809 22:52:28.749203 (+ 58us) client_negotiation.cc:240] Negotiation successful
0809 22:52:28.749305 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":752,"thread_start_us":605,"threads_started":1}
I20260809 22:52:28.798650 18391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:52:28.788053 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.113.131:58547 (local address 127.15.113.129:45313)
0809 22:52:28.788437 (+ 384us) server_negotiation.cc:206] Beginning negotiation
0809 22:52:28.788441 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 22:52:28.789770 (+ 1329us) server_negotiation.cc:407] Connection header received
0809 22:52:28.790079 (+ 309us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:52:28.790085 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:52:28.790184 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:52:28.790252 (+ 68us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 22:52:28.791657 (+ 1405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:52:28.793225 (+ 1568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:52:28.796219 (+ 2994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:52:28.797030 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:52:28.797169 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:52:28.797804 (+ 635us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 22:52:28.797926 (+ 122us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 22:52:28.797987 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0809 22:52:28.798178 (+ 191us) server_negotiation.cc:299] Negotiation successful
0809 22:52:28.798251 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":246}
I20260809 22:52:28.799712 18425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:52:28.788385 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.129:45313 (local address 127.15.113.131:58547)
0809 22:52:28.789621 (+ 1236us) negotiation.cc:107] Waiting for socket to connect
0809 22:52:28.789643 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0809 22:52:28.789911 (+ 268us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:52:28.790503 (+ 592us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:52:28.790507 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:52:28.790520 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 22:52:28.791442 (+ 922us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:52:28.791458 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:52:28.793397 (+ 1939us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:52:28.793402 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:52:28.796045 (+ 2643us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:52:28.796061 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:52:28.797262 (+ 1201us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:52:28.797269 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:52:28.797517 (+ 248us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:52:28.797585 (+ 68us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 22:52:28.798064 (+ 479us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 22:52:28.798073 (+ 9us) client_negotiation.cc:769] Sending connection context
0809 22:52:28.798896 (+ 823us) client_negotiation.cc:240] Negotiation successful
0809 22:52:28.799025 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1014,"thread_start_us":2740,"threads_started":1}
I20260809 22:52:28.839977 17540 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:42384:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 22:52:28.840291 17540 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:42384:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 22:52:28.844367 17540 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 152c29ffd1d34d12b577dbc76022feba: Sending DeleteTablet for 3 replicas of tablet f61d736329a84372a0814ab03a66f100
I20260809 22:52:28.845265 17963 tablet_service.cc:1514] Processing DeleteTablet for tablet f61d736329a84372a0814ab03a66f100 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:52:28 UTC) from {username='kudu', principal='kudu/127.15.113.190@KRBTEST.COM'} at 127.0.0.1:42956
I20260809 22:52:28.845463 17751 tablet_service.cc:1514] Processing DeleteTablet for tablet f61d736329a84372a0814ab03a66f100 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:52:28 UTC) from {username='kudu', principal='kudu/127.15.113.190@KRBTEST.COM'} at 127.0.0.1:36642
I20260809 22:52:28.846694 18178 tablet_service.cc:1514] Processing DeleteTablet for tablet f61d736329a84372a0814ab03a66f100 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:52:28 UTC) from {username='kudu', principal='kudu/127.15.113.190@KRBTEST.COM'} at 127.0.0.1:56810
I20260809 22:52:28.857249 18431 tablet_replica.cc:330] T f61d736329a84372a0814ab03a66f100 P 6d8a1e80214f4c0192bf6f3f3c7d5b19: stopping tablet replica
I20260809 22:52:28.857511 18428 tablet_replica.cc:330] T f61d736329a84372a0814ab03a66f100 P 7cc5e8e5db8840fda0686859e4c61823: stopping tablet replica
I20260809 22:52:28.857684 18429 tablet_replica.cc:330] T f61d736329a84372a0814ab03a66f100 P 610b38e390064672bc4c03a90c4bb702: stopping tablet replica
I20260809 22:52:28.857846 18428 raft_consensus.cc:2229] T f61d736329a84372a0814ab03a66f100 P 7cc5e8e5db8840fda0686859e4c61823 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 22:52:28.858088 18429 raft_consensus.cc:2229] T f61d736329a84372a0814ab03a66f100 P 610b38e390064672bc4c03a90c4bb702 [term 1 LEADER]: Raft consensus shutting down.
I20260809 22:52:28.858232 18428 raft_consensus.cc:2258] T f61d736329a84372a0814ab03a66f100 P 7cc5e8e5db8840fda0686859e4c61823 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 22:52:28.858556 18429 raft_consensus.cc:2258] T f61d736329a84372a0814ab03a66f100 P 610b38e390064672bc4c03a90c4bb702 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 22:52:28.858772 18431 raft_consensus.cc:2229] T f61d736329a84372a0814ab03a66f100 P 6d8a1e80214f4c0192bf6f3f3c7d5b19 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 22:52:28.859275 18431 raft_consensus.cc:2258] T f61d736329a84372a0814ab03a66f100 P 6d8a1e80214f4c0192bf6f3f3c7d5b19 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 22:52:28.859467 18428 ts_tablet_manager.cc:1905] T f61d736329a84372a0814ab03a66f100 P 7cc5e8e5db8840fda0686859e4c61823: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 22:52:28.860292 18429 ts_tablet_manager.cc:1905] T f61d736329a84372a0814ab03a66f100 P 610b38e390064672bc4c03a90c4bb702: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 22:52:28.860404 18431 ts_tablet_manager.cc:1905] T f61d736329a84372a0814ab03a66f100 P 6d8a1e80214f4c0192bf6f3f3c7d5b19: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 22:52:28.868091 18428 ts_tablet_manager.cc:1918] T f61d736329a84372a0814ab03a66f100 P 7cc5e8e5db8840fda0686859e4c61823: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 22:52:28.868383 18428 log.cc:1196] T f61d736329a84372a0814ab03a66f100 P 7cc5e8e5db8840fda0686859e4c61823: Deleting WAL directory at /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315935617919-15814-0/minicluster-data/ts-1/wal/wals/f61d736329a84372a0814ab03a66f100
I20260809 22:52:28.868492 18429 ts_tablet_manager.cc:1918] T f61d736329a84372a0814ab03a66f100 P 610b38e390064672bc4c03a90c4bb702: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 22:52:28.868741 18429 log.cc:1196] T f61d736329a84372a0814ab03a66f100 P 610b38e390064672bc4c03a90c4bb702: Deleting WAL directory at /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315935617919-15814-0/minicluster-data/ts-0/wal/wals/f61d736329a84372a0814ab03a66f100
I20260809 22:52:28.869305 18428 ts_tablet_manager.cc:1939] T f61d736329a84372a0814ab03a66f100 P 7cc5e8e5db8840fda0686859e4c61823: Deleting consensus metadata
I20260809 22:52:28.869424 18429 ts_tablet_manager.cc:1939] T f61d736329a84372a0814ab03a66f100 P 610b38e390064672bc4c03a90c4bb702: Deleting consensus metadata
I20260809 22:52:28.869508 18431 ts_tablet_manager.cc:1918] T f61d736329a84372a0814ab03a66f100 P 6d8a1e80214f4c0192bf6f3f3c7d5b19: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 22:52:28.869745 18431 log.cc:1196] T f61d736329a84372a0814ab03a66f100 P 6d8a1e80214f4c0192bf6f3f3c7d5b19: Deleting WAL directory at /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315935617919-15814-0/minicluster-data/ts-2/wal/wals/f61d736329a84372a0814ab03a66f100
I20260809 22:52:28.870460 17519 catalog_manager.cc:4841] TS 7cc5e8e5db8840fda0686859e4c61823 (127.15.113.130:37287): tablet f61d736329a84372a0814ab03a66f100 (table test-table [id=8780eb4f4f234a7c955dcf149974b623]) successfully deleted
I20260809 22:52:28.870867 17519 catalog_manager.cc:4841] TS 610b38e390064672bc4c03a90c4bb702 (127.15.113.129:45313): tablet f61d736329a84372a0814ab03a66f100 (table test-table [id=8780eb4f4f234a7c955dcf149974b623]) successfully deleted
I20260809 22:52:28.871989 18431 ts_tablet_manager.cc:1939] T f61d736329a84372a0814ab03a66f100 P 6d8a1e80214f4c0192bf6f3f3c7d5b19: Deleting consensus metadata
I20260809 22:52:28.873039 17516 catalog_manager.cc:4841] TS 6d8a1e80214f4c0192bf6f3f3c7d5b19 (127.15.113.131:36465): tablet f61d736329a84372a0814ab03a66f100 (table test-table [id=8780eb4f4f234a7c955dcf149974b623]) successfully deleted
Aug 09 22:52:28 dist-test-slave-ncp9 krb5kdc[17476](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315947, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.15.113.129@KRBTEST.COM
I20260809 22:52:28.887494 18391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:52:28.857767 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47318 (local address 127.15.113.129:45313)
0809 22:52:28.857942 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0809 22:52:28.857948 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:52:28.866843 (+ 8895us) server_negotiation.cc:407] Connection header received
0809 22:52:28.867181 (+ 338us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:52:28.867187 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:52:28.867296 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:52:28.867372 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:52:28.870536 (+ 3164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:52:28.871700 (+ 1164us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:52:28.873908 (+ 2208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:52:28.874373 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:52:28.874468 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:52:28.880631 (+ 6163us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:52:28.880644 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:52:28.880660 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:52:28.880721 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:52:28.884287 (+ 3566us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:52:28.885191 (+ 904us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:52:28.885198 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:52:28.885201 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:52:28.885345 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:52:28.885763 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:52:28.885766 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:52:28.885769 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:52:28.886191 (+ 422us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:52:28.886485 (+ 294us) server_negotiation.cc:1035] Waiting for connection context
0809 22:52:28.886829 (+ 344us) server_negotiation.cc:299] Negotiation successful
0809 22:52:28.887044 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
W20260809 22:52:28.888540 17846 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47318
I20260809 22:52:28.910498 18437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:52:28.892424 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51432 (local address 127.15.113.190:38741)
0809 22:52:28.895593 (+ 3169us) server_negotiation.cc:206] Beginning negotiation
0809 22:52:28.895599 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:52:28.896154 (+ 555us) server_negotiation.cc:407] Connection header received
0809 22:52:28.896314 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:52:28.896318 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:52:28.896382 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:52:28.896467 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:52:28.898696 (+ 2229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:52:28.899608 (+ 912us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:52:28.901357 (+ 1749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:52:28.901882 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:52:28.901979 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:52:28.904974 (+ 2995us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:52:28.904982 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:52:28.904985 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:52:28.905019 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:52:28.907672 (+ 2653us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:52:28.908580 (+ 908us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:52:28.908584 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:52:28.908587 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:52:28.908742 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:52:28.909143 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:52:28.909147 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:52:28.909149 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:52:28.909409 (+ 260us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:52:28.909617 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0809 22:52:28.909928 (+ 311us) server_negotiation.cc:299] Negotiation successful
0809 22:52:28.910132 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2981,"thread_start_us":2676,"threads_started":1}
W20260809 22:52:28.911058 17540 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51432
I20260809 22:52:28.913414 15814 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskItYPJR/build/asan/bin/kudu pid 17644
I20260809 22:52:28.927165 18391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:52:28.914284 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47332 (local address 127.15.113.129:45313)
0809 22:52:28.914440 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0809 22:52:28.914446 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:52:28.914860 (+ 414us) server_negotiation.cc:407] Connection header received
0809 22:52:28.914975 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:52:28.914980 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:52:28.915069 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:52:28.915169 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:52:28.916176 (+ 1007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:52:28.917252 (+ 1076us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:52:28.918452 (+ 1200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:52:28.919122 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:52:28.919249 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:52:28.921318 (+ 2069us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:52:28.921325 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:52:28.921330 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:52:28.921368 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:52:28.924685 (+ 3317us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:52:28.925308 (+ 623us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:52:28.925314 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:52:28.925318 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:52:28.925472 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:52:28.925776 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:52:28.925781 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:52:28.925783 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:52:28.926163 (+ 380us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:52:28.926418 (+ 255us) server_negotiation.cc:1035] Waiting for connection context
0809 22:52:28.926584 (+ 166us) server_negotiation.cc:299] Negotiation successful
0809 22:52:28.926813 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
W20260809 22:52:28.927745 17846 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47332
W20260809 22:52:28.928279 15814 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskItYPJR/build/asan/bin/kudu pid 17644: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 22:52:28.928443 15814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskItYPJR/build/asan/bin/kudu with pid 17644
I20260809 22:52:28.961737 15814 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskItYPJR/build/asan/bin/kudu pid 17862
Aug 09 22:52:28 dist-test-slave-ncp9 krb5kdc[17476](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315947, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.15.113.130@KRBTEST.COM
I20260809 22:52:28.976774 18393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:52:28.962519 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49086 (local address 127.15.113.130:37287)
0809 22:52:28.962725 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0809 22:52:28.962731 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:52:28.962751 (+ 20us) server_negotiation.cc:407] Connection header received
0809 22:52:28.962948 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:52:28.962954 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:52:28.963024 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:52:28.963109 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:52:28.964306 (+ 1197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:52:28.965183 (+ 877us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:52:28.966426 (+ 1243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:52:28.966875 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:52:28.966986 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:52:28.971002 (+ 4016us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:52:28.971019 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:52:28.971034 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:52:28.971076 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:52:28.974057 (+ 2981us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:52:28.974698 (+ 641us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:52:28.974707 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:52:28.974709 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:52:28.974850 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:52:28.975217 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:52:28.975221 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:52:28.975223 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:52:28.975596 (+ 373us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:52:28.975888 (+ 292us) server_negotiation.cc:1035] Waiting for connection context
0809 22:52:28.976208 (+ 320us) server_negotiation.cc:299] Negotiation successful
0809 22:52:28.976441 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
W20260809 22:52:28.977468 18071 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:49086
W20260809 22:52:28.978121 15814 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskItYPJR/build/asan/bin/kudu pid 17862: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 22:52:28.978286 15814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskItYPJR/build/asan/bin/kudu with pid 17862
I20260809 22:52:29.006783 15814 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskItYPJR/build/asan/bin/kudu pid 18082
Aug 09 22:52:29 dist-test-slave-ncp9 krb5kdc[17476](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315947, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.15.113.131@KRBTEST.COM
I20260809 22:52:29.022747 18402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:52:29.007681 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37728 (local address 127.15.113.131:36465)
0809 22:52:29.007923 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0809 22:52:29.007929 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:52:29.007950 (+ 21us) server_negotiation.cc:407] Connection header received
0809 22:52:29.008047 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:52:29.008052 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:52:29.008149 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:52:29.008245 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:52:29.009596 (+ 1351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:52:29.010456 (+ 860us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:52:29.011770 (+ 1314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:52:29.012240 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:52:29.012361 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:52:29.016769 (+ 4408us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:52:29.016781 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:52:29.016797 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:52:29.016829 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:52:29.019919 (+ 3090us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:52:29.020730 (+ 811us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:52:29.020737 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:52:29.020740 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:52:29.020857 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:52:29.021295 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:52:29.021298 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:52:29.021300 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:52:29.021669 (+ 369us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:52:29.021920 (+ 251us) server_negotiation.cc:1035] Waiting for connection context
0809 22:52:29.022271 (+ 351us) server_negotiation.cc:299] Negotiation successful
0809 22:52:29.022423 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
W20260809 22:52:29.023394 18282 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:37728
W20260809 22:52:29.024065 15814 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskItYPJR/build/asan/bin/kudu pid 18082: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 22:52:29.024232 15814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskItYPJR/build/asan/bin/kudu with pid 18082
I20260809 22:52:29.052496 15814 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskItYPJR/build/asan/bin/kudu pid 17492
I20260809 22:52:29.053643 17599 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:52:29.168205 15814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskItYPJR/build/asan/bin/kudu with pid 17492
2026-08-09T22:52:29Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (4899 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315935617919-15814-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 22:52:29 dist-test-slave-ncp9 krb5kdc[18445](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 22:52:29 dist-test-slave-ncp9 krb5kdc[18445](info): set up 2 sockets
Aug 09 22:52:29 dist-test-slave-ncp9 krb5kdc[18445](info): commencing operation
krb5kdc: starting...
W20260809 22:52:30.376415 15814 mini_kdc.cc:121] Time spent starting KDC: real 1.158s user 0.001s sys 0.021s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315935617919-15814-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315935617919-15814-0/krb5kdc/test-user.keytab.
Aug 09 22:52:30 dist-test-slave-ncp9 krb5kdc[18445](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315950, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T22:52:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T22:52:30Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.113.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.113.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.113.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.190.keytab.
Entry for principal kudu/127.15.113.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.190.keytab.
I20260809 22:52:30.562379 15814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskItYPJR/build/asan/bin/kudu
/tmp/dist-test-taskItYPJR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315935617919-15814-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315935617919-15814-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315935617919-15814-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315935617919-15814-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.113.190:45749
--webserver_interface=127.15.113.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.190.keytab
--principal=kudu/127.15.113.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.113.148:38967
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.113.190:45749
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315935617919-15814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315935617919-15814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315935617919-15814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:52:30.882745 18461 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:52:30.883011 18461 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:52:30.883107 18461 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:52:30.890023 18461 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 22:52:30.890125 18461 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260809 22:52:30.890173 18461 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:52:30.890233 18461 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:52:30.890282 18461 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 22:52:30.890342 18461 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 22:52:30.899811 18461 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.113.148:38967
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315935617919-15814-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315935617919-15814-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.113.190:45749
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.190.keytab
--principal=kudu/127.15.113.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.113.190:45749
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315935617919-15814-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.113.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315935617919-15814-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:52:30.900955 18461 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:52:30.902204 18461 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:52:30.913362 18472 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:52:30.913441 18475 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:52:30.913647 18471 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:52:30.914337 18461 server_base.cc:1034] running on GCE node
I20260809 22:52:30.915006 18461 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:52:30.916306 18461 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:52:30.917510 18461 hybrid_clock.cc:648] HybridClock initialized: now 1786315950917443 us; error 35 us; skew 500 ppm
I20260809 22:52:30.917825 18461 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:52:30 dist-test-slave-ncp9 krb5kdc[18445](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315950, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.113.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:52:30.921033 18461 init.cc:377] Logged in from keytab as kudu/127.15.113.190@KRBTEST.COM (short username kudu)
I20260809 22:52:30.922374 18461 webserver.cc:460] Webserver started at http://127.15.113.190:36045/ using document root <none> and password file <none>
I20260809 22:52:30.923056 18461 fs_manager.cc:359] Metadata directory not provided
I20260809 22:52:30.923161 18461 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:52:30.923476 18461 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:52:30.926437 18461 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315935617919-15814-0/minicluster-data/master-0/data/instance:
uuid: "72c0d959ee5c41298a2ea15ccc17437f"
format_stamp: "Formatted at 2026-08-09 22:52:30 on dist-test-slave-ncp9"
server_key: "81c0f122a718670cd753d31975e42724"
server_key_iv: "ceb399042656a893794a42f500fe4b89"
server_key_version: "encryptionkey@0"
I20260809 22:52:30.927208 18461 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315935617919-15814-0/minicluster-data/master-0/wal/instance:
uuid: "72c0d959ee5c41298a2ea15ccc17437f"
format_stamp: "Formatted at 2026-08-09 22:52:30 on dist-test-slave-ncp9"
server_key: "81c0f122a718670cd753d31975e42724"
server_key_iv: "ceb399042656a893794a42f500fe4b89"
server_key_version: "encryptionkey@0"
I20260809 22:52:30.932199 18461 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.004s
I20260809 22:52:30.935559 18486 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:52:30.936774 18461 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 22:52:30.936964 18461 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315935617919-15814-0/minicluster-data/master-0/data,/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315935617919-15814-0/minicluster-data/master-0/wal
uuid: "72c0d959ee5c41298a2ea15ccc17437f"
format_stamp: "Formatted at 2026-08-09 22:52:30 on dist-test-slave-ncp9"
server_key: "81c0f122a718670cd753d31975e42724"
server_key_iv: "ceb399042656a893794a42f500fe4b89"
server_key_version: "encryptionkey@0"
I20260809 22:52:30.937170 18461 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315935617919-15814-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315935617919-15814-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315935617919-15814-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:52:30.968453 18461 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:52:30.972283 18461 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:52:30.972536 18461 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:52:30.992450 18461 rpc_server.cc:307] RPC server started. Bound to: 127.15.113.190:45749
I20260809 22:52:30.992457 18588 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.113.190:45749 every 8 connection(s)
I20260809 22:52:30.993729 18461 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315935617919-15814-0/minicluster-data/master-0/data/info.pb
I20260809 22:52:30.997401 18591 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:52:30.998234 15814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskItYPJR/build/asan/bin/kudu as pid 18461
I20260809 22:52:30.998427 15814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315935617919-15814-0/minicluster-data/master-0/wal/instance
I20260809 22:52:30.999258 15814 external_mini_cluster.cc:1428] Setting key abeadb088d324d26fd79f9335fce0d0e
I20260809 22:52:31.011001 18591 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 72c0d959ee5c41298a2ea15ccc17437f: Bootstrap starting.
Aug 09 22:52:31 dist-test-slave-ncp9 krb5kdc[18445](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315950, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.113.190@KRBTEST.COM
I20260809 22:52:31.014438 18591 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 72c0d959ee5c41298a2ea15ccc17437f: Neither blocks nor log segments found. Creating new log.
I20260809 22:52:31.015739 18591 log.cc:824] T 00000000000000000000000000000000 P 72c0d959ee5c41298a2ea15ccc17437f: Log is configured to *not* fsync() on all Append() calls
I20260809 22:52:31.018859 18591 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 72c0d959ee5c41298a2ea15ccc17437f: No bootstrap required, opened a new log
I20260809 22:52:31.023036 18596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:52:31.001402 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60632 (local address 127.15.113.190:45749)
0809 22:52:31.002251 (+ 849us) server_negotiation.cc:206] Beginning negotiation
0809 22:52:31.002263 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 22:52:31.002303 (+ 40us) server_negotiation.cc:407] Connection header received
0809 22:52:31.003872 (+ 1569us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:52:31.003893 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:52:31.004260 (+ 367us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:52:31.004579 (+ 319us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:52:31.006251 (+ 1672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:52:31.007531 (+ 1280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:52:31.009314 (+ 1783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:52:31.009950 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:52:31.010081 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:52:31.014471 (+ 4390us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:52:31.014479 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:52:31.014492 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:52:31.014534 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:52:31.018531 (+ 3997us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:52:31.019266 (+ 735us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:52:31.019274 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:52:31.019279 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:52:31.019479 (+ 200us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:52:31.019877 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:52:31.019881 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:52:31.019883 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:52:31.020460 (+ 577us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:52:31.020791 (+ 331us) server_negotiation.cc:1035] Waiting for connection context
0809 22:52:31.021255 (+ 464us) server_negotiation.cc:299] Negotiation successful
0809 22:52:31.021715 (+ 460us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":622,"thread_start_us":396,"threads_started":1}
I20260809 22:52:31.027119 18591 raft_consensus.cc:348] T 00000000000000000000000000000000 P 72c0d959ee5c41298a2ea15ccc17437f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "72c0d959ee5c41298a2ea15ccc17437f" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 45749 } }
I20260809 22:52:31.027429 18591 raft_consensus.cc:374] T 00000000000000000000000000000000 P 72c0d959ee5c41298a2ea15ccc17437f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:52:31.027514 18591 raft_consensus.cc:729] T 00000000000000000000000000000000 P 72c0d959ee5c41298a2ea15ccc17437f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 72c0d959ee5c41298a2ea15ccc17437f, State: Initialized, Role: FOLLOWER
I20260809 22:52:31.028008 18591 consensus_queue.cc:260] T 00000000000000000000000000000000 P 72c0d959ee5c41298a2ea15ccc17437f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "72c0d959ee5c41298a2ea15ccc17437f" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 45749 } }
I20260809 22:52:31.028218 18591 raft_consensus.cc:388] T 00000000000000000000000000000000 P 72c0d959ee5c41298a2ea15ccc17437f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 22:52:31.028316 18591 raft_consensus.cc:482] T 00000000000000000000000000000000 P 72c0d959ee5c41298a2ea15ccc17437f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 22:52:31.028435 18591 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 72c0d959ee5c41298a2ea15ccc17437f [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:52:31.030468 18591 raft_consensus.cc:504] T 00000000000000000000000000000000 P 72c0d959ee5c41298a2ea15ccc17437f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "72c0d959ee5c41298a2ea15ccc17437f" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 45749 } }
I20260809 22:52:31.030887 18591 leader_election.cc:302] T 00000000000000000000000000000000 P 72c0d959ee5c41298a2ea15ccc17437f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 72c0d959ee5c41298a2ea15ccc17437f; no voters:
I20260809 22:52:31.031417 18591 leader_election.cc:288] T 00000000000000000000000000000000 P 72c0d959ee5c41298a2ea15ccc17437f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 22:52:31.031586 18598 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 72c0d959ee5c41298a2ea15ccc17437f [term 1 FOLLOWER]: Leader election won for term 1
I20260809 22:52:31.032346 18598 raft_consensus.cc:686] T 00000000000000000000000000000000 P 72c0d959ee5c41298a2ea15ccc17437f [term 1 LEADER]: Becoming Leader. State: Replica: 72c0d959ee5c41298a2ea15ccc17437f, State: Running, Role: LEADER
I20260809 22:52:31.033020 18598 consensus_queue.cc:237] T 00000000000000000000000000000000 P 72c0d959ee5c41298a2ea15ccc17437f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "72c0d959ee5c41298a2ea15ccc17437f" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 45749 } }
I20260809 22:52:31.033480 18591 sys_catalog.cc:564] T 00000000000000000000000000000000 P 72c0d959ee5c41298a2ea15ccc17437f [sys.catalog]: configured and running, proceeding with master startup.
I20260809 22:52:31.036038 18603 sys_catalog.cc:455] T 00000000000000000000000000000000 P 72c0d959ee5c41298a2ea15ccc17437f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 72c0d959ee5c41298a2ea15ccc17437f. Latest consensus state: current_term: 1 leader_uuid: "72c0d959ee5c41298a2ea15ccc17437f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "72c0d959ee5c41298a2ea15ccc17437f" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 45749 } } }
I20260809 22:52:31.036094 18600 sys_catalog.cc:455] T 00000000000000000000000000000000 P 72c0d959ee5c41298a2ea15ccc17437f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "72c0d959ee5c41298a2ea15ccc17437f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "72c0d959ee5c41298a2ea15ccc17437f" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 45749 } } }
I20260809 22:52:31.036497 18603 sys_catalog.cc:458] T 00000000000000000000000000000000 P 72c0d959ee5c41298a2ea15ccc17437f [sys.catalog]: This master's current role is: LEADER
I20260809 22:52:31.036506 18600 sys_catalog.cc:458] T 00000000000000000000000000000000 P 72c0d959ee5c41298a2ea15ccc17437f [sys.catalog]: This master's current role is: LEADER
I20260809 22:52:31.041899 18610 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 22:52:31.047240 18610 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 22:52:31.056407 18610 catalog_manager.cc:1347] Generated new cluster ID: 15ee7024b4df47e4b85f66743b4a4dc5
I20260809 22:52:31.056500 18610 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 22:52:31.070535 18610 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 22:52:31.072330 18610 catalog_manager.cc:1504] Loading token signing keys...
I20260809 22:52:31.085211 18610 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 72c0d959ee5c41298a2ea15ccc17437f: Generated new TSK 0
I20260809 22:52:31.086243 18610 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.113.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.113.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.113.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.129.keytab.
Entry for principal kudu/127.15.113.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.129.keytab.
I20260809 22:52:31.144258 15814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskItYPJR/build/asan/bin/kudu
/tmp/dist-test-taskItYPJR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315935617919-15814-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315935617919-15814-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315935617919-15814-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315935617919-15814-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.113.129:0
--local_ip_for_outbound_sockets=127.15.113.129
--webserver_interface=127.15.113.129
--webserver_port=0
--tserver_master_addrs=127.15.113.190:45749
--keytab_file=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.129.keytab
--principal=kudu/127.15.113.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.113.148:38967
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315935617919-15814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315935617919-15814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315935617919-15814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:52:31.467034 18627 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:52:31.467307 18627 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:52:31.467455 18627 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:52:31.474299 18627 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:52:31.474406 18627 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:52:31.474524 18627 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260809 22:52:31.474591 18627 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 22:52:31.474689 18627 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.113.129
I20260809 22:52:31.483989 18627 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.113.148:38967
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315935617919-15814-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315935617919-15814-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.129.keytab
--principal=kudu/127.15.113.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.113.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315935617919-15814-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.113.129
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.15.113.190:45749
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.113.129
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315935617919-15814-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:52:31.485193 18627 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:52:31.486445 18627 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:52:31.498572 18635 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:52:31.498620 18636 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:52:31.498703 18638 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:52:31.499724 18627 server_base.cc:1034] running on GCE node
I20260809 22:52:31.500272 18627 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:52:31.501161 18627 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:52:31.502353 18627 hybrid_clock.cc:648] HybridClock initialized: now 1786315951502309 us; error 39 us; skew 500 ppm
I20260809 22:52:31.502630 18627 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:52:31 dist-test-slave-ncp9 krb5kdc[18445](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315951, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.113.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:52:31.505564 18627 init.cc:377] Logged in from keytab as kudu/127.15.113.129@KRBTEST.COM (short username kudu)
I20260809 22:52:31.506922 18627 webserver.cc:460] Webserver started at http://127.15.113.129:33171/ using document root <none> and password file <none>
I20260809 22:52:31.507543 18627 fs_manager.cc:359] Metadata directory not provided
I20260809 22:52:31.507668 18627 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:52:31.507969 18627 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:52:31.510915 18627 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315935617919-15814-0/minicluster-data/ts-0/data/instance:
uuid: "fa95fcf2448c4166902c56d01332630c"
format_stamp: "Formatted at 2026-08-09 22:52:31 on dist-test-slave-ncp9"
server_key: "224a26c45813af05ab071570344b7038"
server_key_iv: "c9fde3e96169fc8711f477a01bc8a1fc"
server_key_version: "encryptionkey@0"
I20260809 22:52:31.511626 18627 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315935617919-15814-0/minicluster-data/ts-0/wal/instance:
uuid: "fa95fcf2448c4166902c56d01332630c"
format_stamp: "Formatted at 2026-08-09 22:52:31 on dist-test-slave-ncp9"
server_key: "224a26c45813af05ab071570344b7038"
server_key_iv: "c9fde3e96169fc8711f477a01bc8a1fc"
server_key_version: "encryptionkey@0"
I20260809 22:52:31.516527 18627 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260809 22:52:31.520004 18649 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:52:31.521292 18627 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 22:52:31.521423 18627 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315935617919-15814-0/minicluster-data/ts-0/data,/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315935617919-15814-0/minicluster-data/ts-0/wal
uuid: "fa95fcf2448c4166902c56d01332630c"
format_stamp: "Formatted at 2026-08-09 22:52:31 on dist-test-slave-ncp9"
server_key: "224a26c45813af05ab071570344b7038"
server_key_iv: "c9fde3e96169fc8711f477a01bc8a1fc"
server_key_version: "encryptionkey@0"
I20260809 22:52:31.521582 18627 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315935617919-15814-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315935617919-15814-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315935617919-15814-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:52:31.539323 18627 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:52:31.543226 18627 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:52:31.543491 18627 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:52:31.546176 18627 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:52:31.546296 18627 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:52:31.546425 18627 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:52:31.546555 18627 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.001s sys 0.000s
Aug 09 22:52:31 dist-test-slave-ncp9 krb5kdc[18445](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315951, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.113.129@KRBTEST.COM for kudu/127.15.113.190@KRBTEST.COM
I20260809 22:52:31.568673 18666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:52:31.547225 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:45749 (local address 127.15.113.129:53045)
0809 22:52:31.548044 (+ 819us) negotiation.cc:107] Waiting for socket to connect
0809 22:52:31.548095 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0809 22:52:31.550096 (+ 2001us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:52:31.550819 (+ 723us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:52:31.550827 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:52:31.551405 (+ 578us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:52:31.552258 (+ 853us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:52:31.552288 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:52:31.553699 (+ 1411us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:52:31.553706 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:52:31.555797 (+ 2091us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:52:31.555816 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:52:31.556606 (+ 790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:52:31.556615 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:52:31.556957 (+ 342us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:52:31.557738 (+ 781us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:52:31.557770 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:52:31.561066 (+ 3296us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:52:31.564087 (+ 3021us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:52:31.564093 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:52:31.564104 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:52:31.564701 (+ 597us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:52:31.565103 (+ 402us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:52:31.565107 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:52:31.565113 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:52:31.565298 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:52:31.565898 (+ 600us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:52:31.565905 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:52:31.566401 (+ 496us) client_negotiation.cc:769] Sending connection context
0809 22:52:31.566694 (+ 293us) client_negotiation.cc:240] Negotiation successful
0809 22:52:31.567179 (+ 485us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":562,"thread_start_us":367,"threads_started":1}
I20260809 22:52:31.569600 18662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:52:31.546544 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.113.129:53045 (local address 127.15.113.190:45749)
0809 22:52:31.547023 (+ 479us) server_negotiation.cc:206] Beginning negotiation
0809 22:52:31.547030 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:52:31.548195 (+ 1165us) server_negotiation.cc:407] Connection header received
0809 22:52:31.550374 (+ 2179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:52:31.550380 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:52:31.550459 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:52:31.550554 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:52:31.552505 (+ 1951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:52:31.553498 (+ 993us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:52:31.555978 (+ 2480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:52:31.556399 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:52:31.556490 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:52:31.561300 (+ 4810us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:52:31.561307 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:52:31.561311 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:52:31.561351 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:52:31.563915 (+ 2564us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:52:31.564818 (+ 903us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:52:31.564824 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:52:31.564828 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:52:31.564993 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:52:31.565412 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:52:31.565417 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:52:31.565421 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:52:31.565781 (+ 360us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:52:31.565924 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0809 22:52:31.569059 (+ 3135us) server_negotiation.cc:299] Negotiation successful
0809 22:52:31.569271 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":350,"thread_start_us":248,"threads_started":1}
I20260809 22:52:31.589032 18627 rpc_server.cc:307] RPC server started. Bound to: 127.15.113.129:35739
I20260809 22:52:31.589093 18830 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.113.129:35739 every 8 connection(s)
I20260809 22:52:31.590456 18662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:52:31.575177 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.113.129:51371 (local address 127.15.113.190:45749)
0809 22:52:31.575341 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0809 22:52:31.575346 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:52:31.576199 (+ 853us) server_negotiation.cc:407] Connection header received
0809 22:52:31.576418 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:52:31.576423 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:52:31.576500 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:52:31.576597 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:52:31.578136 (+ 1539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:52:31.579145 (+ 1009us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:52:31.581252 (+ 2107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:52:31.581754 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:52:31.581874 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:52:31.584404 (+ 2530us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:52:31.584412 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:52:31.584417 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:52:31.584456 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:52:31.587654 (+ 3198us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:52:31.588544 (+ 890us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:52:31.588550 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:52:31.588553 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:52:31.588726 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:52:31.589253 (+ 527us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:52:31.589257 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:52:31.589260 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:52:31.589540 (+ 280us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:52:31.589666 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0809 22:52:31.590084 (+ 418us) server_negotiation.cc:299] Negotiation successful
0809 22:52:31.590245 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260809 22:52:31.590519 18627 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315935617919-15814-0/minicluster-data/ts-0/data/info.pb
I20260809 22:52:31.590766 18777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:52:31.575356 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:45749 (local address 127.15.113.129:51371)
0809 22:52:31.576103 (+ 747us) negotiation.cc:107] Waiting for socket to connect
0809 22:52:31.576117 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 22:52:31.576239 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:52:31.576651 (+ 412us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:52:31.576656 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:52:31.577427 (+ 771us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:52:31.577954 (+ 527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:52:31.577974 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:52:31.579304 (+ 1330us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:52:31.579310 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:52:31.581109 (+ 1799us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:52:31.581133 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:52:31.581992 (+ 859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:52:31.581999 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:52:31.582209 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:52:31.582884 (+ 675us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:52:31.582910 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:52:31.584165 (+ 1255us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:52:31.587832 (+ 3667us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:52:31.587837 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:52:31.587841 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:52:31.588422 (+ 581us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:52:31.588862 (+ 440us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:52:31.588868 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:52:31.588872 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:52:31.589060 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:52:31.589659 (+ 599us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:52:31.589664 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:52:31.589864 (+ 200us) client_negotiation.cc:769] Sending connection context
0809 22:52:31.590012 (+ 148us) client_negotiation.cc:240] Negotiation successful
0809 22:52:31.590315 (+ 303us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":628,"thread_start_us":501,"threads_started":1}
I20260809 22:52:31.592152 15814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskItYPJR/build/asan/bin/kudu as pid 18627
I20260809 22:52:31.592347 15814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315935617919-15814-0/minicluster-data/ts-0/wal/instance
I20260809 22:52:31.593168 15814 external_mini_cluster.cc:1428] Setting key 08600cee7239852f812d3f5a1e615a12
I20260809 22:52:31.592974 18831 heartbeater.cc:347] Connected to a master server at 127.15.113.190:45749
I20260809 22:52:31.593976 18831 heartbeater.cc:464] Registering TS with master...
I20260809 22:52:31.594734 18831 heartbeater.cc:511] Master 127.15.113.190:45749 requested a full tablet report, sending...
I20260809 22:52:31.596479 18514 ts_manager.cc:194] Registered new tserver with Master: fa95fcf2448c4166902c56d01332630c (127.15.113.129:35739)
I20260809 22:52:31.597859 18514 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.113.129@KRBTEST.COM'} at 127.15.113.129:53045
WARNING: no policy specified for kudu/127.15.113.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.113.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.113.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.130.keytab.
Entry for principal kudu/127.15.113.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.130.keytab.
I20260809 22:52:31.635810 15814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskItYPJR/build/asan/bin/kudu
/tmp/dist-test-taskItYPJR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315935617919-15814-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315935617919-15814-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315935617919-15814-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315935617919-15814-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.113.130:0
--local_ip_for_outbound_sockets=127.15.113.130
--webserver_interface=127.15.113.130
--webserver_port=0
--tserver_master_addrs=127.15.113.190:45749
--keytab_file=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.130.keytab
--principal=kudu/127.15.113.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.113.148:38967
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315935617919-15814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315935617919-15814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315935617919-15814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:52:31.956770 18837 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:52:31.957062 18837 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:52:31.957213 18837 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:52:31.964184 18837 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:52:31.964290 18837 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:52:31.964418 18837 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260809 22:52:31.964484 18837 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 22:52:31.964592 18837 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.113.130
I20260809 22:52:31.974144 18837 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.113.148:38967
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315935617919-15814-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315935617919-15814-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.130.keytab
--principal=kudu/127.15.113.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.113.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315935617919-15814-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.113.130
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.15.113.190:45749
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.113.130
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315935617919-15814-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:52:31.975379 18837 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:52:31.976681 18837 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:52:31.989265 18843 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:52:31.989301 18846 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:52:31.989264 18844 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:52:31.989967 18837 server_base.cc:1034] running on GCE node
I20260809 22:52:31.990509 18837 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:52:31.991338 18837 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:52:31.992530 18837 hybrid_clock.cc:648] HybridClock initialized: now 1786315951992485 us; error 39 us; skew 500 ppm
I20260809 22:52:31.992815 18837 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:52:31 dist-test-slave-ncp9 krb5kdc[18445](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315951, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.113.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:52:31.995790 18837 init.cc:377] Logged in from keytab as kudu/127.15.113.130@KRBTEST.COM (short username kudu)
I20260809 22:52:31.997264 18837 webserver.cc:460] Webserver started at http://127.15.113.130:38319/ using document root <none> and password file <none>
I20260809 22:52:31.997881 18837 fs_manager.cc:359] Metadata directory not provided
I20260809 22:52:31.997989 18837 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:52:31.998291 18837 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:52:32.001077 18837 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315935617919-15814-0/minicluster-data/ts-1/data/instance:
uuid: "0c75510608d24ff78855de400475c524"
format_stamp: "Formatted at 2026-08-09 22:52:31 on dist-test-slave-ncp9"
server_key: "f420698720b389c9d36f51f9f74e4fdc"
server_key_iv: "c7fb75631039caa22f32ab8d53148ec0"
server_key_version: "encryptionkey@0"
I20260809 22:52:32.001853 18837 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315935617919-15814-0/minicluster-data/ts-1/wal/instance:
uuid: "0c75510608d24ff78855de400475c524"
format_stamp: "Formatted at 2026-08-09 22:52:31 on dist-test-slave-ncp9"
server_key: "f420698720b389c9d36f51f9f74e4fdc"
server_key_iv: "c7fb75631039caa22f32ab8d53148ec0"
server_key_version: "encryptionkey@0"
I20260809 22:52:32.006731 18837 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 22:52:32.009966 18861 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:52:32.011176 18837 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 22:52:32.011323 18837 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315935617919-15814-0/minicluster-data/ts-1/data,/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315935617919-15814-0/minicluster-data/ts-1/wal
uuid: "0c75510608d24ff78855de400475c524"
format_stamp: "Formatted at 2026-08-09 22:52:31 on dist-test-slave-ncp9"
server_key: "f420698720b389c9d36f51f9f74e4fdc"
server_key_iv: "c7fb75631039caa22f32ab8d53148ec0"
server_key_version: "encryptionkey@0"
I20260809 22:52:32.011488 18837 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315935617919-15814-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315935617919-15814-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315935617919-15814-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:52:32.033308 18837 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:52:32.036972 18837 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:52:32.037230 18837 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:52:32.039621 18837 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:52:32.039726 18837 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:52:32.039831 18837 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:52:32.039911 18837 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 22:52:32 dist-test-slave-ncp9 krb5kdc[18445](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315951, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.113.130@KRBTEST.COM for kudu/127.15.113.190@KRBTEST.COM
I20260809 22:52:32.062157 18877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:52:32.040738 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:45749 (local address 127.15.113.130:60631)
0809 22:52:32.041586 (+ 848us) negotiation.cc:107] Waiting for socket to connect
0809 22:52:32.041711 (+ 125us) client_negotiation.cc:174] Beginning negotiation
0809 22:52:32.043627 (+ 1916us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:52:32.044483 (+ 856us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:52:32.044491 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:52:32.045217 (+ 726us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:52:32.045991 (+ 774us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:52:32.046016 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:52:32.047321 (+ 1305us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:52:32.047326 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:52:32.049444 (+ 2118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:52:32.049457 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:52:32.050405 (+ 948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:52:32.050413 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:52:32.050717 (+ 304us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:52:32.051622 (+ 905us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:52:32.051648 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:52:32.054220 (+ 2572us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:52:32.057623 (+ 3403us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:52:32.057632 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:52:32.057650 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:52:32.058021 (+ 371us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:52:32.058460 (+ 439us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:52:32.058465 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:52:32.058469 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:52:32.058702 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:52:32.059395 (+ 693us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:52:32.059406 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:52:32.059810 (+ 404us) client_negotiation.cc:769] Sending connection context
0809 22:52:32.060129 (+ 319us) client_negotiation.cc:240] Negotiation successful
0809 22:52:32.060615 (+ 486us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":593,"thread_start_us":381,"threads_started":1}
I20260809 22:52:32.062650 18662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:52:32.040181 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.113.130:60631 (local address 127.15.113.190:45749)
0809 22:52:32.040395 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0809 22:52:32.040402 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:52:32.041840 (+ 1438us) server_negotiation.cc:407] Connection header received
0809 22:52:32.044010 (+ 2170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:52:32.044018 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:52:32.044112 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:52:32.044228 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:52:32.046232 (+ 2004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:52:32.047168 (+ 936us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:52:32.049571 (+ 2403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:52:32.050072 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:52:32.050197 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:52:32.054439 (+ 4242us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:52:32.054444 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:52:32.054449 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:52:32.054489 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:52:32.057505 (+ 3016us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:52:32.058145 (+ 640us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:52:32.058151 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:52:32.058155 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:52:32.058323 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:52:32.058851 (+ 528us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:52:32.058857 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:52:32.058860 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:52:32.059236 (+ 376us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:52:32.059410 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0809 22:52:32.062079 (+ 2669us) server_negotiation.cc:299] Negotiation successful
0809 22:52:32.062329 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260809 22:52:32.082733 18662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:52:32.068166 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.113.130:40539 (local address 127.15.113.190:45749)
0809 22:52:32.068342 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0809 22:52:32.068347 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:52:32.069658 (+ 1311us) server_negotiation.cc:407] Connection header received
0809 22:52:32.069827 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:52:32.069830 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:52:32.069885 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:52:32.069956 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:52:32.071092 (+ 1136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:52:32.071843 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:52:32.074003 (+ 2160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:52:32.074486 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:52:32.074606 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:52:32.076856 (+ 2250us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:52:32.076862 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:52:32.076868 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:52:32.076925 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:52:32.079762 (+ 2837us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:52:32.080536 (+ 774us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:52:32.080543 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:52:32.080547 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:52:32.080739 (+ 192us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:52:32.081231 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:52:32.081236 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:52:32.081239 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:52:32.081633 (+ 394us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:52:32.081766 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0809 22:52:32.082212 (+ 446us) server_negotiation.cc:299] Negotiation successful
0809 22:52:32.082416 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260809 22:52:32.082877 19001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:52:32.068261 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:45749 (local address 127.15.113.130:40539)
0809 22:52:32.069538 (+ 1277us) negotiation.cc:107] Waiting for socket to connect
0809 22:52:32.069559 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0809 22:52:32.069718 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:52:32.069996 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:52:32.070001 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:52:32.070552 (+ 551us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:52:32.070943 (+ 391us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:52:32.070956 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:52:32.071966 (+ 1010us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:52:32.071972 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:52:32.073842 (+ 1870us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:52:32.073862 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:52:32.074718 (+ 856us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:52:32.074726 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:52:32.074946 (+ 220us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:52:32.075575 (+ 629us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:52:32.075598 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:52:32.076661 (+ 1063us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:52:32.079881 (+ 3220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:52:32.079886 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:52:32.079890 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:52:32.080418 (+ 528us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:52:32.080915 (+ 497us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:52:32.080921 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:52:32.080924 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:52:32.081102 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:52:32.081749 (+ 647us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:52:32.081758 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:52:32.081935 (+ 177us) client_negotiation.cc:769] Sending connection context
0809 22:52:32.082053 (+ 118us) client_negotiation.cc:240] Negotiation successful
0809 22:52:32.082418 (+ 365us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1084,"thread_start_us":887,"threads_started":1}
I20260809 22:52:32.084432 18837 rpc_server.cc:307] RPC server started. Bound to: 127.15.113.130:34115
I20260809 22:52:32.084473 19051 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.113.130:34115 every 8 connection(s)
I20260809 22:52:32.085836 18837 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315935617919-15814-0/minicluster-data/ts-1/data/info.pb
I20260809 22:52:32.087895 19052 heartbeater.cc:347] Connected to a master server at 127.15.113.190:45749
I20260809 22:52:32.088076 19052 heartbeater.cc:464] Registering TS with master...
I20260809 22:52:32.088613 19052 heartbeater.cc:511] Master 127.15.113.190:45749 requested a full tablet report, sending...
I20260809 22:52:32.090029 18514 ts_manager.cc:194] Registered new tserver with Master: 0c75510608d24ff78855de400475c524 (127.15.113.130:34115)
I20260809 22:52:32.090744 18514 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.113.130@KRBTEST.COM'} at 127.15.113.130:60631
I20260809 22:52:32.093312 15814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskItYPJR/build/asan/bin/kudu as pid 18837
I20260809 22:52:32.093461 15814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315935617919-15814-0/minicluster-data/ts-1/wal/instance
I20260809 22:52:32.093907 15814 external_mini_cluster.cc:1428] Setting key de0a43ad0a99a3e3f9457bd3dd6465f6
WARNING: no policy specified for kudu/127.15.113.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.113.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.113.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.131.keytab.
Entry for principal kudu/127.15.113.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.131.keytab.
I20260809 22:52:32.135911 15814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskItYPJR/build/asan/bin/kudu
/tmp/dist-test-taskItYPJR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315935617919-15814-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315935617919-15814-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315935617919-15814-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315935617919-15814-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.113.131:0
--local_ip_for_outbound_sockets=127.15.113.131
--webserver_interface=127.15.113.131
--webserver_port=0
--tserver_master_addrs=127.15.113.190:45749
--keytab_file=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.131.keytab
--principal=kudu/127.15.113.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.113.148:38967
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315935617919-15814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315935617919-15814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315935617919-15814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:52:32.454438 19058 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:52:32.454715 19058 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:52:32.454869 19058 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:52:32.462002 19058 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:52:32.462109 19058 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:52:32.462255 19058 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260809 22:52:32.462324 19058 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 22:52:32.462424 19058 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.113.131
I20260809 22:52:32.471951 19058 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.113.148:38967
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315935617919-15814-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315935617919-15814-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.131.keytab
--principal=kudu/127.15.113.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.113.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315935617919-15814-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.113.131
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.15.113.190:45749
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.113.131
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315935617919-15814-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:52:32.473186 19058 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:52:32.474436 19058 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:52:32.486546 19067 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:52:32.486580 19070 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:52:32.486507 19068 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:52:32.487613 19058 server_base.cc:1034] running on GCE node
I20260809 22:52:32.488164 19058 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:52:32.489073 19058 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:52:32.490260 19058 hybrid_clock.cc:648] HybridClock initialized: now 1786315952490222 us; error 33 us; skew 500 ppm
I20260809 22:52:32.490535 19058 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:52:32 dist-test-slave-ncp9 krb5kdc[18445](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315952, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.113.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:52:32.493433 19058 init.cc:377] Logged in from keytab as kudu/127.15.113.131@KRBTEST.COM (short username kudu)
I20260809 22:52:32.494813 19058 webserver.cc:460] Webserver started at http://127.15.113.131:36389/ using document root <none> and password file <none>
I20260809 22:52:32.495425 19058 fs_manager.cc:359] Metadata directory not provided
I20260809 22:52:32.495533 19058 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:52:32.495832 19058 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:52:32.498601 19058 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315935617919-15814-0/minicluster-data/ts-2/data/instance:
uuid: "8e8fcf7e414b4304a18a79967f8a53ba"
format_stamp: "Formatted at 2026-08-09 22:52:32 on dist-test-slave-ncp9"
server_key: "b440418c31f97812a2e7d28c02442e3d"
server_key_iv: "07f249423e3c4e1afcde15f5fb9a4650"
server_key_version: "encryptionkey@0"
I20260809 22:52:32.499313 19058 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315935617919-15814-0/minicluster-data/ts-2/wal/instance:
uuid: "8e8fcf7e414b4304a18a79967f8a53ba"
format_stamp: "Formatted at 2026-08-09 22:52:32 on dist-test-slave-ncp9"
server_key: "b440418c31f97812a2e7d28c02442e3d"
server_key_iv: "07f249423e3c4e1afcde15f5fb9a4650"
server_key_version: "encryptionkey@0"
I20260809 22:52:32.504128 19058 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.001s
I20260809 22:52:32.507350 19084 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:52:32.508589 19058 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260809 22:52:32.508744 19058 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315935617919-15814-0/minicluster-data/ts-2/data,/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315935617919-15814-0/minicluster-data/ts-2/wal
uuid: "8e8fcf7e414b4304a18a79967f8a53ba"
format_stamp: "Formatted at 2026-08-09 22:52:32 on dist-test-slave-ncp9"
server_key: "b440418c31f97812a2e7d28c02442e3d"
server_key_iv: "07f249423e3c4e1afcde15f5fb9a4650"
server_key_version: "encryptionkey@0"
I20260809 22:52:32.508946 19058 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315935617919-15814-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315935617919-15814-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315935617919-15814-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:52:32.531769 19058 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:52:32.535669 19058 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:52:32.535926 19058 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:52:32.538499 19058 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:52:32.538592 19058 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:52:32.538712 19058 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:52:32.538801 19058 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 22:52:32 dist-test-slave-ncp9 krb5kdc[18445](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315952, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.113.131@KRBTEST.COM for kudu/127.15.113.190@KRBTEST.COM
I20260809 22:52:32.561549 18662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:52:32.538974 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.113.131:57779 (local address 127.15.113.190:45749)
0809 22:52:32.539135 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0809 22:52:32.539140 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:52:32.540812 (+ 1672us) server_negotiation.cc:407] Connection header received
0809 22:52:32.543744 (+ 2932us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:52:32.543750 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:52:32.543841 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:52:32.543948 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:52:32.546054 (+ 2106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:52:32.547016 (+ 962us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:52:32.549875 (+ 2859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:52:32.550378 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:52:32.550512 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:52:32.554858 (+ 4346us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:52:32.554865 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:52:32.554869 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:52:32.554913 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:52:32.557880 (+ 2967us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:52:32.558882 (+ 1002us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:52:32.558888 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:52:32.558891 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:52:32.559052 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:52:32.559616 (+ 564us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:52:32.559622 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:52:32.559625 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:52:32.559992 (+ 367us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:52:32.560147 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0809 22:52:32.560968 (+ 821us) server_negotiation.cc:299] Negotiation successful
0809 22:52:32.561195 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260809 22:52:32.562528 19098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:52:32.539667 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:45749 (local address 127.15.113.131:57779)
0809 22:52:32.540613 (+ 946us) negotiation.cc:107] Waiting for socket to connect
0809 22:52:32.540669 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0809 22:52:32.543361 (+ 2692us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:52:32.544228 (+ 867us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:52:32.544236 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:52:32.544992 (+ 756us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:52:32.545857 (+ 865us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:52:32.545882 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:52:32.547217 (+ 1335us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:52:32.547223 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:52:32.549717 (+ 2494us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:52:32.549737 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:52:32.550680 (+ 943us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:52:32.550690 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:52:32.550968 (+ 278us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:52:32.551623 (+ 655us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:52:32.551657 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:52:32.554614 (+ 2957us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:52:32.558027 (+ 3413us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:52:32.558035 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:52:32.558054 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:52:32.558737 (+ 683us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:52:32.559170 (+ 433us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:52:32.559175 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:52:32.559179 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:52:32.559491 (+ 312us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:52:32.560211 (+ 720us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:52:32.560217 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:52:32.560535 (+ 318us) client_negotiation.cc:769] Sending connection context
0809 22:52:32.560770 (+ 235us) client_negotiation.cc:240] Negotiation successful
0809 22:52:32.561347 (+ 577us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":656,"thread_start_us":398,"threads_started":1}
I20260809 22:52:32.580196 18662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:52:32.567934 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.113.131:44375 (local address 127.15.113.190:45749)
0809 22:52:32.568116 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0809 22:52:32.568122 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:52:32.568672 (+ 550us) server_negotiation.cc:407] Connection header received
0809 22:52:32.568826 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:52:32.568830 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:52:32.568886 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:52:32.568990 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:52:32.570288 (+ 1298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:52:32.571210 (+ 922us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:52:32.572491 (+ 1281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:52:32.572994 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:52:32.573098 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:52:32.574724 (+ 1626us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:52:32.574729 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:52:32.574732 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:52:32.574761 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:52:32.577288 (+ 2527us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:52:32.578117 (+ 829us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:52:32.578123 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:52:32.578126 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:52:32.578258 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:52:32.578728 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:52:32.578734 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:52:32.578737 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:52:32.579099 (+ 362us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:52:32.579266 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0809 22:52:32.579661 (+ 395us) server_negotiation.cc:299] Negotiation successful
0809 22:52:32.579869 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260809 22:52:32.580348 19196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:52:32.567936 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:45749 (local address 127.15.113.131:44375)
0809 22:52:32.568569 (+ 633us) negotiation.cc:107] Waiting for socket to connect
0809 22:52:32.568589 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 22:52:32.568722 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:52:32.569052 (+ 330us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:52:32.569057 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:52:32.569700 (+ 643us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:52:32.570135 (+ 435us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:52:32.570148 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:52:32.571323 (+ 1175us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:52:32.571329 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:52:32.572356 (+ 1027us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:52:32.572370 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:52:32.573207 (+ 837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:52:32.573215 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:52:32.573352 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:52:32.573880 (+ 528us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:52:32.573896 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:52:32.574547 (+ 651us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:52:32.577408 (+ 2861us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:52:32.577413 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:52:32.577418 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:52:32.578007 (+ 589us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:52:32.578389 (+ 382us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:52:32.578398 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:52:32.578402 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:52:32.578568 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:52:32.579234 (+ 666us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:52:32.579237 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:52:32.579392 (+ 155us) client_negotiation.cc:769] Sending connection context
0809 22:52:32.579526 (+ 134us) client_negotiation.cc:240] Negotiation successful
0809 22:52:32.579861 (+ 335us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":476,"thread_start_us":335,"threads_started":1}
I20260809 22:52:32.592487 19058 rpc_server.cc:307] RPC server started. Bound to: 127.15.113.131:44931
I20260809 22:52:32.592507 19277 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.113.131:44931 every 8 connection(s)
I20260809 22:52:32.593735 19058 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315935617919-15814-0/minicluster-data/ts-2/data/info.pb
I20260809 22:52:32.594446 15814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskItYPJR/build/asan/bin/kudu as pid 19058
I20260809 22:52:32.594624 15814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315935617919-15814-0/minicluster-data/ts-2/wal/instance
I20260809 22:52:32.595132 15814 external_mini_cluster.cc:1428] Setting key 9e6a6ba61bd3523888cdf8a6286e0417
I20260809 22:52:32.595656 19278 heartbeater.cc:347] Connected to a master server at 127.15.113.190:45749
I20260809 22:52:32.595866 19278 heartbeater.cc:464] Registering TS with master...
I20260809 22:52:32.596441 19278 heartbeater.cc:511] Master 127.15.113.190:45749 requested a full tablet report, sending...
I20260809 22:52:32.597815 18513 ts_manager.cc:194] Registered new tserver with Master: 8e8fcf7e414b4304a18a79967f8a53ba (127.15.113.131:44931)
I20260809 22:52:32.598642 18513 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.113.131@KRBTEST.COM'} at 127.15.113.131:57779
I20260809 22:52:32.600206 18831 heartbeater.cc:503] Master 127.15.113.190:45749 was elected leader, sending a full tablet report...
I20260809 22:52:32.609625 15814 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 09 22:52:32 dist-test-slave-ncp9 krb5kdc[18445](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315952, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 09 22:52:32 dist-test-slave-ncp9 krb5kdc[18445](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315952, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.113.190@KRBTEST.COM
I20260809 22:52:32.668615 18662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:52:32.646672 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60638 (local address 127.15.113.190:45749)
0809 22:52:32.646900 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0809 22:52:32.646906 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:52:32.647148 (+ 242us) server_negotiation.cc:407] Connection header received
0809 22:52:32.647343 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:52:32.647366 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:52:32.647433 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:52:32.647520 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:52:32.649060 (+ 1540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:52:32.649842 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:52:32.651072 (+ 1230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:52:32.651495 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:52:32.651611 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:52:32.663668 (+ 12057us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:52:32.663673 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:52:32.663677 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:52:32.663713 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:52:32.666355 (+ 2642us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:52:32.666986 (+ 631us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:52:32.666990 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:52:32.666992 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:52:32.667104 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:52:32.667453 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:52:32.667456 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:52:32.667458 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:52:32.667727 (+ 269us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:52:32.667911 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0809 22:52:32.668129 (+ 218us) server_negotiation.cc:299] Negotiation successful
0809 22:52:32.668299 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260809 22:52:32.673295 18513 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60638:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 22:52:32.680164 18513 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 22:52:32.702858 19303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:52:32.692329 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38374 (local address 127.15.113.130:34115)
0809 22:52:32.694278 (+ 1949us) server_negotiation.cc:206] Beginning negotiation
0809 22:52:32.694289 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 22:52:32.694321 (+ 32us) server_negotiation.cc:407] Connection header received
0809 22:52:32.694420 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:52:32.694427 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:52:32.694691 (+ 264us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:52:32.696039 (+ 1348us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:52:32.696249 (+ 210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:52:32.697662 (+ 1413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:52:32.700299 (+ 2637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:52:32.701665 (+ 1366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:52:32.701847 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:52:32.702036 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0809 22:52:32.702397 (+ 361us) server_negotiation.cc:299] Negotiation successful
0809 22:52:32.702568 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1772,"thread_start_us":1633,"threads_started":1}
I20260809 22:52:32.702966 19302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:52:32.692709 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.131:44931 (local address 127.0.0.1:48732)
0809 22:52:32.695041 (+ 2332us) negotiation.cc:107] Waiting for socket to connect
0809 22:52:32.695060 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 22:52:32.695174 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:52:32.696156 (+ 982us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:52:32.696161 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:52:32.696189 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:52:32.696574 (+ 385us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:52:32.696586 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:52:32.698304 (+ 1718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:52:32.698309 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:52:32.700295 (+ 1986us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:52:32.700309 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:52:32.702064 (+ 1755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:52:32.702071 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:52:32.702221 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:52:32.702245 (+ 24us) client_negotiation.cc:769] Sending connection context
0809 22:52:32.702303 (+ 58us) client_negotiation.cc:240] Negotiation successful
0809 22:52:32.702536 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2175,"mutex_wait_us":34,"thread_start_us":909,"threads_started":1}
I20260809 22:52:32.703909 19305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:52:32.692122 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.130:34115 (local address 127.0.0.1:38374)
0809 22:52:32.693465 (+ 1343us) negotiation.cc:107] Waiting for socket to connect
0809 22:52:32.693515 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0809 22:52:32.693695 (+ 180us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:52:32.695341 (+ 1646us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:52:32.695349 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:52:32.695378 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:52:32.695814 (+ 436us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:52:32.695828 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:52:32.697845 (+ 2017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:52:32.697854 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:52:32.700146 (+ 2292us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:52:32.700161 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:52:32.701856 (+ 1695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:52:32.701861 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:52:32.702071 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:52:32.702180 (+ 109us) client_negotiation.cc:769] Sending connection context
0809 22:52:32.702320 (+ 140us) client_negotiation.cc:240] Negotiation successful
0809 22:52:32.703524 (+ 1204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1069,"mutex_wait_us":52,"thread_start_us":2616,"threads_started":1}
I20260809 22:52:32.704150 19306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:52:32.693362 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48732 (local address 127.15.113.131:44931)
0809 22:52:32.694751 (+ 1389us) server_negotiation.cc:206] Beginning negotiation
0809 22:52:32.694760 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 22:52:32.695286 (+ 526us) server_negotiation.cc:407] Connection header received
0809 22:52:32.695385 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:52:32.695393 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:52:32.695649 (+ 256us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:52:32.695847 (+ 198us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:52:32.696791 (+ 944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:52:32.698153 (+ 1362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:52:32.700450 (+ 2297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:52:32.701837 (+ 1387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:52:32.703491 (+ 1654us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:52:32.703611 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0809 22:52:32.703680 (+ 69us) server_negotiation.cc:299] Negotiation successful
0809 22:52:32.703837 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1190,"thread_start_us":975,"threads_started":1}
I20260809 22:52:32.704303 19307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:52:32.695405 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34790 (local address 127.15.113.129:35739)
0809 22:52:32.697106 (+ 1701us) server_negotiation.cc:206] Beginning negotiation
0809 22:52:32.697114 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 22:52:32.697141 (+ 27us) server_negotiation.cc:407] Connection header received
0809 22:52:32.697227 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:52:32.697234 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:52:32.697428 (+ 194us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:52:32.697599 (+ 171us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:52:32.698233 (+ 634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:52:32.699726 (+ 1493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:52:32.701527 (+ 1801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:52:32.702898 (+ 1371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:52:32.703066 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:52:32.703246 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0809 22:52:32.703748 (+ 502us) server_negotiation.cc:299] Negotiation successful
0809 22:52:32.703895 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1549,"thread_start_us":1410,"threads_started":1}
I20260809 22:52:32.704311 19308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:52:32.695679 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.129:35739 (local address 127.0.0.1:34790)
0809 22:52:32.696194 (+ 515us) negotiation.cc:107] Waiting for socket to connect
0809 22:52:32.696211 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 22:52:32.696348 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:52:32.697615 (+ 1267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:52:32.697620 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:52:32.697646 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:52:32.698038 (+ 392us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:52:32.698050 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:52:32.699880 (+ 1830us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:52:32.699886 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:52:32.701381 (+ 1495us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:52:32.701394 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:52:32.703387 (+ 1993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:52:32.703394 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:52:32.703564 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:52:32.703594 (+ 30us) client_negotiation.cc:769] Sending connection context
0809 22:52:32.703691 (+ 97us) client_negotiation.cc:240] Negotiation successful
0809 22:52:32.703811 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":400,"thread_start_us":402,"threads_started":1}
I20260809 22:52:32.707470 19161 tablet_service.cc:1467] Processing CreateTablet for tablet 53dcd318d9c24d79acb67622a42e83ec (DEFAULT_TABLE table=test-table [id=d488e6510c1e466eaab14daa65952b12]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:52:32.707749 18732 tablet_service.cc:1467] Processing CreateTablet for tablet 53dcd318d9c24d79acb67622a42e83ec (DEFAULT_TABLE table=test-table [id=d488e6510c1e466eaab14daa65952b12]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:52:32.707752 18947 tablet_service.cc:1467] Processing CreateTablet for tablet 53dcd318d9c24d79acb67622a42e83ec (DEFAULT_TABLE table=test-table [id=d488e6510c1e466eaab14daa65952b12]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:52:32.708976 19161 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 53dcd318d9c24d79acb67622a42e83ec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:52:32.709132 18732 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 53dcd318d9c24d79acb67622a42e83ec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:52:32.709162 18947 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 53dcd318d9c24d79acb67622a42e83ec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:52:32.721946 19310 tablet_bootstrap.cc:492] T 53dcd318d9c24d79acb67622a42e83ec P 8e8fcf7e414b4304a18a79967f8a53ba: Bootstrap starting.
I20260809 22:52:32.723280 19312 tablet_bootstrap.cc:492] T 53dcd318d9c24d79acb67622a42e83ec P 0c75510608d24ff78855de400475c524: Bootstrap starting.
I20260809 22:52:32.724050 19311 tablet_bootstrap.cc:492] T 53dcd318d9c24d79acb67622a42e83ec P fa95fcf2448c4166902c56d01332630c: Bootstrap starting.
I20260809 22:52:32.726404 19310 tablet_bootstrap.cc:654] T 53dcd318d9c24d79acb67622a42e83ec P 8e8fcf7e414b4304a18a79967f8a53ba: Neither blocks nor log segments found. Creating new log.
I20260809 22:52:32.727512 19311 tablet_bootstrap.cc:654] T 53dcd318d9c24d79acb67622a42e83ec P fa95fcf2448c4166902c56d01332630c: Neither blocks nor log segments found. Creating new log.
I20260809 22:52:32.727543 19312 tablet_bootstrap.cc:654] T 53dcd318d9c24d79acb67622a42e83ec P 0c75510608d24ff78855de400475c524: Neither blocks nor log segments found. Creating new log.
I20260809 22:52:32.727720 19310 log.cc:824] T 53dcd318d9c24d79acb67622a42e83ec P 8e8fcf7e414b4304a18a79967f8a53ba: Log is configured to *not* fsync() on all Append() calls
I20260809 22:52:32.728644 19312 log.cc:824] T 53dcd318d9c24d79acb67622a42e83ec P 0c75510608d24ff78855de400475c524: Log is configured to *not* fsync() on all Append() calls
I20260809 22:52:32.728729 19311 log.cc:824] T 53dcd318d9c24d79acb67622a42e83ec P fa95fcf2448c4166902c56d01332630c: Log is configured to *not* fsync() on all Append() calls
I20260809 22:52:32.731244 19310 tablet_bootstrap.cc:492] T 53dcd318d9c24d79acb67622a42e83ec P 8e8fcf7e414b4304a18a79967f8a53ba: No bootstrap required, opened a new log
I20260809 22:52:32.731528 19310 ts_tablet_manager.cc:1397] T 53dcd318d9c24d79acb67622a42e83ec P 8e8fcf7e414b4304a18a79967f8a53ba: Time spent bootstrapping tablet: real 0.011s user 0.006s sys 0.000s
I20260809 22:52:32.731981 19311 tablet_bootstrap.cc:492] T 53dcd318d9c24d79acb67622a42e83ec P fa95fcf2448c4166902c56d01332630c: No bootstrap required, opened a new log
I20260809 22:52:32.732239 19311 ts_tablet_manager.cc:1397] T 53dcd318d9c24d79acb67622a42e83ec P fa95fcf2448c4166902c56d01332630c: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260809 22:52:32.735062 19312 tablet_bootstrap.cc:492] T 53dcd318d9c24d79acb67622a42e83ec P 0c75510608d24ff78855de400475c524: No bootstrap required, opened a new log
I20260809 22:52:32.735297 19312 ts_tablet_manager.cc:1397] T 53dcd318d9c24d79acb67622a42e83ec P 0c75510608d24ff78855de400475c524: Time spent bootstrapping tablet: real 0.012s user 0.009s sys 0.000s
I20260809 22:52:32.741822 19310 raft_consensus.cc:348] T 53dcd318d9c24d79acb67622a42e83ec P 8e8fcf7e414b4304a18a79967f8a53ba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c75510608d24ff78855de400475c524" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 34115 } } peers { permanent_uuid: "fa95fcf2448c4166902c56d01332630c" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 35739 } } peers { permanent_uuid: "8e8fcf7e414b4304a18a79967f8a53ba" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 44931 } }
I20260809 22:52:32.742269 19310 raft_consensus.cc:374] T 53dcd318d9c24d79acb67622a42e83ec P 8e8fcf7e414b4304a18a79967f8a53ba [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:52:32.742378 19310 raft_consensus.cc:729] T 53dcd318d9c24d79acb67622a42e83ec P 8e8fcf7e414b4304a18a79967f8a53ba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8e8fcf7e414b4304a18a79967f8a53ba, State: Initialized, Role: FOLLOWER
I20260809 22:52:32.742451 19312 raft_consensus.cc:348] T 53dcd318d9c24d79acb67622a42e83ec P 0c75510608d24ff78855de400475c524 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c75510608d24ff78855de400475c524" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 34115 } } peers { permanent_uuid: "fa95fcf2448c4166902c56d01332630c" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 35739 } } peers { permanent_uuid: "8e8fcf7e414b4304a18a79967f8a53ba" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 44931 } }
I20260809 22:52:32.742863 19312 raft_consensus.cc:374] T 53dcd318d9c24d79acb67622a42e83ec P 0c75510608d24ff78855de400475c524 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:52:32.742672 19311 raft_consensus.cc:348] T 53dcd318d9c24d79acb67622a42e83ec P fa95fcf2448c4166902c56d01332630c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c75510608d24ff78855de400475c524" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 34115 } } peers { permanent_uuid: "fa95fcf2448c4166902c56d01332630c" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 35739 } } peers { permanent_uuid: "8e8fcf7e414b4304a18a79967f8a53ba" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 44931 } }
I20260809 22:52:32.742959 19312 raft_consensus.cc:729] T 53dcd318d9c24d79acb67622a42e83ec P 0c75510608d24ff78855de400475c524 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0c75510608d24ff78855de400475c524, State: Initialized, Role: FOLLOWER
I20260809 22:52:32.743060 19311 raft_consensus.cc:374] T 53dcd318d9c24d79acb67622a42e83ec P fa95fcf2448c4166902c56d01332630c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:52:32.743155 19311 raft_consensus.cc:729] T 53dcd318d9c24d79acb67622a42e83ec P fa95fcf2448c4166902c56d01332630c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa95fcf2448c4166902c56d01332630c, State: Initialized, Role: FOLLOWER
I20260809 22:52:32.743002 19310 consensus_queue.cc:260] T 53dcd318d9c24d79acb67622a42e83ec P 8e8fcf7e414b4304a18a79967f8a53ba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c75510608d24ff78855de400475c524" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 34115 } } peers { permanent_uuid: "fa95fcf2448c4166902c56d01332630c" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 35739 } } peers { permanent_uuid: "8e8fcf7e414b4304a18a79967f8a53ba" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 44931 } }
I20260809 22:52:32.743650 19312 consensus_queue.cc:260] T 53dcd318d9c24d79acb67622a42e83ec P 0c75510608d24ff78855de400475c524 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c75510608d24ff78855de400475c524" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 34115 } } peers { permanent_uuid: "fa95fcf2448c4166902c56d01332630c" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 35739 } } peers { permanent_uuid: "8e8fcf7e414b4304a18a79967f8a53ba" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 44931 } }
I20260809 22:52:32.743883 19311 consensus_queue.cc:260] T 53dcd318d9c24d79acb67622a42e83ec P fa95fcf2448c4166902c56d01332630c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c75510608d24ff78855de400475c524" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 34115 } } peers { permanent_uuid: "fa95fcf2448c4166902c56d01332630c" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 35739 } } peers { permanent_uuid: "8e8fcf7e414b4304a18a79967f8a53ba" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 44931 } }
I20260809 22:52:32.744150 19278 heartbeater.cc:503] Master 127.15.113.190:45749 was elected leader, sending a full tablet report...
I20260809 22:52:32.744801 19310 ts_tablet_manager.cc:1428] T 53dcd318d9c24d79acb67622a42e83ec P 8e8fcf7e414b4304a18a79967f8a53ba: Time spent starting tablet: real 0.013s user 0.015s sys 0.000s
I20260809 22:52:32.745839 19052 heartbeater.cc:503] Master 127.15.113.190:45749 was elected leader, sending a full tablet report...
I20260809 22:52:32.746347 19312 ts_tablet_manager.cc:1428] T 53dcd318d9c24d79acb67622a42e83ec P 0c75510608d24ff78855de400475c524: Time spent starting tablet: real 0.011s user 0.000s sys 0.011s
I20260809 22:52:32.748121 19311 ts_tablet_manager.cc:1428] T 53dcd318d9c24d79acb67622a42e83ec P fa95fcf2448c4166902c56d01332630c: Time spent starting tablet: real 0.016s user 0.013s sys 0.001s
I20260809 22:52:32.813172 19326 raft_consensus.cc:482] T 53dcd318d9c24d79acb67622a42e83ec P 0c75510608d24ff78855de400475c524 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 22:52:32.813480 19326 raft_consensus.cc:504] T 53dcd318d9c24d79acb67622a42e83ec P 0c75510608d24ff78855de400475c524 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c75510608d24ff78855de400475c524" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 34115 } } peers { permanent_uuid: "fa95fcf2448c4166902c56d01332630c" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 35739 } } peers { permanent_uuid: "8e8fcf7e414b4304a18a79967f8a53ba" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 44931 } }
I20260809 22:52:32.815179 19326 leader_election.cc:288] T 53dcd318d9c24d79acb67622a42e83ec P 0c75510608d24ff78855de400475c524 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fa95fcf2448c4166902c56d01332630c (127.15.113.129:35739), 8e8fcf7e414b4304a18a79967f8a53ba (127.15.113.131:44931)
I20260809 22:52:32.823561 19306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:52:32.815663 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.113.130:35661 (local address 127.15.113.131:44931)
0809 22:52:32.816021 (+ 358us) server_negotiation.cc:206] Beginning negotiation
0809 22:52:32.816027 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:52:32.816474 (+ 447us) server_negotiation.cc:407] Connection header received
0809 22:52:32.816602 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:52:32.816608 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:52:32.816693 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:52:32.816783 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:52:32.817706 (+ 923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:52:32.818730 (+ 1024us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:52:32.821348 (+ 2618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:52:32.822539 (+ 1191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:52:32.822708 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:52:32.822770 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0809 22:52:32.823199 (+ 429us) server_negotiation.cc:299] Negotiation successful
0809 22:52:32.823323 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":218}
I20260809 22:52:32.823563 19307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:52:32.815649 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.113.130:36153 (local address 127.15.113.129:35739)
0809 22:52:32.815874 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0809 22:52:32.815882 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 22:52:32.816409 (+ 527us) server_negotiation.cc:407] Connection header received
0809 22:52:32.816604 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:52:32.816608 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:52:32.816693 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:52:32.816787 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:52:32.817567 (+ 780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:52:32.818647 (+ 1080us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:52:32.821348 (+ 2701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:52:32.822553 (+ 1205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:52:32.822698 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:52:32.822770 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0809 22:52:32.823199 (+ 429us) server_negotiation.cc:299] Negotiation successful
0809 22:52:32.823323 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260809 22:52:32.823815 19328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:52:32.815395 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.131:44931 (local address 127.15.113.130:35661)
0809 22:52:32.816384 (+ 989us) negotiation.cc:107] Waiting for socket to connect
0809 22:52:32.816399 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 22:52:32.816490 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:52:32.816813 (+ 323us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:52:32.816817 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:52:32.816861 (+ 44us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:52:32.817540 (+ 679us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:52:32.817551 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:52:32.818842 (+ 1291us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:52:32.818847 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:52:32.821203 (+ 2356us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:52:32.821219 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:52:32.822776 (+ 1557us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:52:32.822782 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:52:32.823024 (+ 242us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:52:32.823083 (+ 59us) client_negotiation.cc:769] Sending connection context
0809 22:52:32.823174 (+ 91us) client_negotiation.cc:240] Negotiation successful
0809 22:52:32.823361 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":889,"mutex_wait_us":211,"spinlock_wait_cycles":6912,"thread_start_us":497,"threads_started":1}
I20260809 22:52:32.823838 19329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:52:32.815381 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.129:35739 (local address 127.15.113.130:36153)
0809 22:52:32.816296 (+ 915us) negotiation.cc:107] Waiting for socket to connect
0809 22:52:32.816315 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 22:52:32.816474 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:52:32.816812 (+ 338us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:52:32.816817 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:52:32.816852 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:52:32.817417 (+ 565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:52:32.817433 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:52:32.818770 (+ 1337us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:52:32.818774 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:52:32.821206 (+ 2432us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:52:32.821219 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:52:32.822781 (+ 1562us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:52:32.822786 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:52:32.823029 (+ 243us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:52:32.823078 (+ 49us) client_negotiation.cc:769] Sending connection context
0809 22:52:32.823171 (+ 93us) client_negotiation.cc:240] Negotiation successful
0809 22:52:32.823361 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":777,"mutex_wait_us":214,"spinlock_wait_cycles":1280,"thread_start_us":706,"threads_started":1}
I20260809 22:52:32.824858 18767 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "53dcd318d9c24d79acb67622a42e83ec" candidate_uuid: "0c75510608d24ff78855de400475c524" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa95fcf2448c4166902c56d01332630c" is_pre_election: true
I20260809 22:52:32.824944 19205 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "53dcd318d9c24d79acb67622a42e83ec" candidate_uuid: "0c75510608d24ff78855de400475c524" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e8fcf7e414b4304a18a79967f8a53ba" is_pre_election: true
I20260809 22:52:32.825408 18767 raft_consensus.cc:2454] T 53dcd318d9c24d79acb67622a42e83ec P fa95fcf2448c4166902c56d01332630c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0c75510608d24ff78855de400475c524 in term 0.
I20260809 22:52:32.825420 19205 raft_consensus.cc:2454] T 53dcd318d9c24d79acb67622a42e83ec P 8e8fcf7e414b4304a18a79967f8a53ba [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0c75510608d24ff78855de400475c524 in term 0.
I20260809 22:52:32.826100 18863 leader_election.cc:302] T 53dcd318d9c24d79acb67622a42e83ec P 0c75510608d24ff78855de400475c524 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0c75510608d24ff78855de400475c524, 8e8fcf7e414b4304a18a79967f8a53ba; no voters:
I20260809 22:52:32.826445 19326 raft_consensus.cc:2781] T 53dcd318d9c24d79acb67622a42e83ec P 0c75510608d24ff78855de400475c524 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 22:52:32.826598 19326 raft_consensus.cc:482] T 53dcd318d9c24d79acb67622a42e83ec P 0c75510608d24ff78855de400475c524 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 22:52:32.826699 19326 raft_consensus.cc:3037] T 53dcd318d9c24d79acb67622a42e83ec P 0c75510608d24ff78855de400475c524 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:52:32.828749 19326 raft_consensus.cc:504] T 53dcd318d9c24d79acb67622a42e83ec P 0c75510608d24ff78855de400475c524 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c75510608d24ff78855de400475c524" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 34115 } } peers { permanent_uuid: "fa95fcf2448c4166902c56d01332630c" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 35739 } } peers { permanent_uuid: "8e8fcf7e414b4304a18a79967f8a53ba" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 44931 } }
I20260809 22:52:32.829337 19326 leader_election.cc:288] T 53dcd318d9c24d79acb67622a42e83ec P 0c75510608d24ff78855de400475c524 [CANDIDATE]: Term 1 election: Requested vote from peers fa95fcf2448c4166902c56d01332630c (127.15.113.129:35739), 8e8fcf7e414b4304a18a79967f8a53ba (127.15.113.131:44931)
I20260809 22:52:32.829679 18767 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "53dcd318d9c24d79acb67622a42e83ec" candidate_uuid: "0c75510608d24ff78855de400475c524" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa95fcf2448c4166902c56d01332630c"
I20260809 22:52:32.829769 19205 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "53dcd318d9c24d79acb67622a42e83ec" candidate_uuid: "0c75510608d24ff78855de400475c524" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e8fcf7e414b4304a18a79967f8a53ba"
I20260809 22:52:32.829845 18767 raft_consensus.cc:3037] T 53dcd318d9c24d79acb67622a42e83ec P fa95fcf2448c4166902c56d01332630c [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:52:32.829963 19205 raft_consensus.cc:3037] T 53dcd318d9c24d79acb67622a42e83ec P 8e8fcf7e414b4304a18a79967f8a53ba [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:52:32.831987 18767 raft_consensus.cc:2454] T 53dcd318d9c24d79acb67622a42e83ec P fa95fcf2448c4166902c56d01332630c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0c75510608d24ff78855de400475c524 in term 1.
I20260809 22:52:32.832043 19205 raft_consensus.cc:2454] T 53dcd318d9c24d79acb67622a42e83ec P 8e8fcf7e414b4304a18a79967f8a53ba [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0c75510608d24ff78855de400475c524 in term 1.
I20260809 22:52:32.832371 18867 leader_election.cc:302] T 53dcd318d9c24d79acb67622a42e83ec P 0c75510608d24ff78855de400475c524 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0c75510608d24ff78855de400475c524, fa95fcf2448c4166902c56d01332630c; no voters:
I20260809 22:52:32.832654 19326 raft_consensus.cc:2781] T 53dcd318d9c24d79acb67622a42e83ec P 0c75510608d24ff78855de400475c524 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 22:52:32.833091 19326 raft_consensus.cc:686] T 53dcd318d9c24d79acb67622a42e83ec P 0c75510608d24ff78855de400475c524 [term 1 LEADER]: Becoming Leader. State: Replica: 0c75510608d24ff78855de400475c524, State: Running, Role: LEADER
I20260809 22:52:32.833511 19326 consensus_queue.cc:237] T 53dcd318d9c24d79acb67622a42e83ec P 0c75510608d24ff78855de400475c524 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c75510608d24ff78855de400475c524" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 34115 } } peers { permanent_uuid: "fa95fcf2448c4166902c56d01332630c" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 35739 } } peers { permanent_uuid: "8e8fcf7e414b4304a18a79967f8a53ba" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 44931 } }
W20260809 22:52:32.837502 19053 tablet.cc:2369] T 53dcd318d9c24d79acb67622a42e83ec P 0c75510608d24ff78855de400475c524: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 22:52:32.837814 18513 catalog_manager.cc:5495] T 53dcd318d9c24d79acb67622a42e83ec P 0c75510608d24ff78855de400475c524 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0c75510608d24ff78855de400475c524 (127.15.113.130). New cstate: current_term: 1 leader_uuid: "0c75510608d24ff78855de400475c524" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c75510608d24ff78855de400475c524" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 34115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fa95fcf2448c4166902c56d01332630c" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 35739 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8e8fcf7e414b4304a18a79967f8a53ba" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 44931 } health_report { overall_health: UNKNOWN } } }
W20260809 22:52:32.842989 18832 tablet.cc:2369] T 53dcd318d9c24d79acb67622a42e83ec P fa95fcf2448c4166902c56d01332630c: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 22:52:32.845193 19281 tablet.cc:2369] T 53dcd318d9c24d79acb67622a42e83ec P 8e8fcf7e414b4304a18a79967f8a53ba: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 09 22:52:32 dist-test-slave-ncp9 krb5kdc[18445](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315950, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.113.190@KRBTEST.COM for kudu/127.15.113.190@KRBTEST.COM
I20260809 22:52:32.887421 18662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:52:32.874966 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60648 (local address 127.15.113.190:45749)
0809 22:52:32.875125 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0809 22:52:32.875133 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 22:52:32.875614 (+ 481us) server_negotiation.cc:407] Connection header received
0809 22:52:32.875764 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:52:32.875767 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:52:32.875832 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:52:32.875909 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:52:32.877055 (+ 1146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:52:32.877773 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:52:32.879014 (+ 1241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:52:32.879422 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:52:32.879526 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:52:32.882729 (+ 3203us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:52:32.882733 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:52:32.882736 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:52:32.882764 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:52:32.884967 (+ 2203us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:52:32.885589 (+ 622us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:52:32.885594 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:52:32.885598 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:52:32.885719 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:52:32.886064 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:52:32.886069 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:52:32.886072 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:52:32.886323 (+ 251us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:52:32.886448 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0809 22:52:32.886849 (+ 401us) server_negotiation.cc:299] Negotiation successful
0809 22:52:32.887124 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":87,"server-negotiator.queue_time_us":57}
I20260809 22:52:32.887455 19340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:52:32.874996 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:45749 (local address 127.0.0.1:60648)
0809 22:52:32.875517 (+ 521us) negotiation.cc:107] Waiting for socket to connect
0809 22:52:32.875532 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 22:52:32.875663 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:52:32.875950 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:52:32.875955 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:52:32.876624 (+ 669us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:52:32.876886 (+ 262us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:52:32.876928 (+ 42us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:52:32.877891 (+ 963us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:52:32.877897 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:52:32.878879 (+ 982us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:52:32.878891 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:52:32.879587 (+ 696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:52:32.879593 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:52:32.879742 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:52:32.880336 (+ 594us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:52:32.880353 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:52:32.882546 (+ 2193us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:52:32.885067 (+ 2521us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:52:32.885080 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:52:32.885088 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:52:32.885486 (+ 398us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:52:32.885828 (+ 342us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:52:32.885833 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:52:32.885837 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:52:32.885970 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:52:32.886449 (+ 479us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:52:32.886458 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:52:32.886602 (+ 144us) client_negotiation.cc:769] Sending connection context
0809 22:52:32.886746 (+ 144us) client_negotiation.cc:240] Negotiation successful
0809 22:52:32.886991 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":408,"thread_start_us":287,"threads_started":1}
I20260809 22:52:32.892050 18513 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.15.113.190@KRBTEST.COM'} at 127.0.0.1:60648:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260809 22:52:32.893028 18513 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 22:52:32.899122 18732 tablet_service.cc:1467] Processing CreateTablet for tablet 84ffa14bf2a34a0da42bb1d66764ba4a (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=ad12966b0b834e908ace062f4028c6e3]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 22:52:32.899258 18947 tablet_service.cc:1467] Processing CreateTablet for tablet 84ffa14bf2a34a0da42bb1d66764ba4a (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=ad12966b0b834e908ace062f4028c6e3]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 22:52:32.899443 19161 tablet_service.cc:1467] Processing CreateTablet for tablet 84ffa14bf2a34a0da42bb1d66764ba4a (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=ad12966b0b834e908ace062f4028c6e3]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 22:52:32.900075 18947 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 84ffa14bf2a34a0da42bb1d66764ba4a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:52:32.900077 18732 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 84ffa14bf2a34a0da42bb1d66764ba4a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:52:32.900290 19161 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 84ffa14bf2a34a0da42bb1d66764ba4a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:52:32.905823 19310 tablet_bootstrap.cc:492] T 84ffa14bf2a34a0da42bb1d66764ba4a P 8e8fcf7e414b4304a18a79967f8a53ba: Bootstrap starting.
I20260809 22:52:32.907022 19312 tablet_bootstrap.cc:492] T 84ffa14bf2a34a0da42bb1d66764ba4a P 0c75510608d24ff78855de400475c524: Bootstrap starting.
I20260809 22:52:32.907214 19310 tablet_bootstrap.cc:654] T 84ffa14bf2a34a0da42bb1d66764ba4a P 8e8fcf7e414b4304a18a79967f8a53ba: Neither blocks nor log segments found. Creating new log.
I20260809 22:52:32.907274 19311 tablet_bootstrap.cc:492] T 84ffa14bf2a34a0da42bb1d66764ba4a P fa95fcf2448c4166902c56d01332630c: Bootstrap starting.
I20260809 22:52:32.908627 19312 tablet_bootstrap.cc:654] T 84ffa14bf2a34a0da42bb1d66764ba4a P 0c75510608d24ff78855de400475c524: Neither blocks nor log segments found. Creating new log.
I20260809 22:52:32.908825 19311 tablet_bootstrap.cc:654] T 84ffa14bf2a34a0da42bb1d66764ba4a P fa95fcf2448c4166902c56d01332630c: Neither blocks nor log segments found. Creating new log.
I20260809 22:52:32.909669 19310 tablet_bootstrap.cc:492] T 84ffa14bf2a34a0da42bb1d66764ba4a P 8e8fcf7e414b4304a18a79967f8a53ba: No bootstrap required, opened a new log
I20260809 22:52:32.909865 19310 ts_tablet_manager.cc:1397] T 84ffa14bf2a34a0da42bb1d66764ba4a P 8e8fcf7e414b4304a18a79967f8a53ba: Time spent bootstrapping tablet: real 0.004s user 0.004s sys 0.000s
I20260809 22:52:32.910588 19310 raft_consensus.cc:348] T 84ffa14bf2a34a0da42bb1d66764ba4a P 8e8fcf7e414b4304a18a79967f8a53ba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa95fcf2448c4166902c56d01332630c" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 35739 } } peers { permanent_uuid: "0c75510608d24ff78855de400475c524" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 34115 } } peers { permanent_uuid: "8e8fcf7e414b4304a18a79967f8a53ba" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 44931 } }
I20260809 22:52:32.910712 19312 tablet_bootstrap.cc:492] T 84ffa14bf2a34a0da42bb1d66764ba4a P 0c75510608d24ff78855de400475c524: No bootstrap required, opened a new log
I20260809 22:52:32.910787 19310 raft_consensus.cc:374] T 84ffa14bf2a34a0da42bb1d66764ba4a P 8e8fcf7e414b4304a18a79967f8a53ba [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:52:32.910848 19312 ts_tablet_manager.cc:1397] T 84ffa14bf2a34a0da42bb1d66764ba4a P 0c75510608d24ff78855de400475c524: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.001s
I20260809 22:52:32.910893 19310 raft_consensus.cc:729] T 84ffa14bf2a34a0da42bb1d66764ba4a P 8e8fcf7e414b4304a18a79967f8a53ba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8e8fcf7e414b4304a18a79967f8a53ba, State: Initialized, Role: FOLLOWER
I20260809 22:52:32.911017 19311 tablet_bootstrap.cc:492] T 84ffa14bf2a34a0da42bb1d66764ba4a P fa95fcf2448c4166902c56d01332630c: No bootstrap required, opened a new log
I20260809 22:52:32.911180 19311 ts_tablet_manager.cc:1397] T 84ffa14bf2a34a0da42bb1d66764ba4a P fa95fcf2448c4166902c56d01332630c: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.001s
I20260809 22:52:32.911155 19310 consensus_queue.cc:260] T 84ffa14bf2a34a0da42bb1d66764ba4a P 8e8fcf7e414b4304a18a79967f8a53ba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa95fcf2448c4166902c56d01332630c" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 35739 } } peers { permanent_uuid: "0c75510608d24ff78855de400475c524" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 34115 } } peers { permanent_uuid: "8e8fcf7e414b4304a18a79967f8a53ba" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 44931 } }
I20260809 22:52:32.911619 19312 raft_consensus.cc:348] T 84ffa14bf2a34a0da42bb1d66764ba4a P 0c75510608d24ff78855de400475c524 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa95fcf2448c4166902c56d01332630c" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 35739 } } peers { permanent_uuid: "0c75510608d24ff78855de400475c524" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 34115 } } peers { permanent_uuid: "8e8fcf7e414b4304a18a79967f8a53ba" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 44931 } }
I20260809 22:52:32.911794 19310 ts_tablet_manager.cc:1428] T 84ffa14bf2a34a0da42bb1d66764ba4a P 8e8fcf7e414b4304a18a79967f8a53ba: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260809 22:52:32.911852 19312 raft_consensus.cc:374] T 84ffa14bf2a34a0da42bb1d66764ba4a P 0c75510608d24ff78855de400475c524 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:52:32.911609 19324 tablet_replica.cc:439] T 84ffa14bf2a34a0da42bb1d66764ba4a P 8e8fcf7e414b4304a18a79967f8a53ba: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa95fcf2448c4166902c56d01332630c" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 35739 } } peers { permanent_uuid: "0c75510608d24ff78855de400475c524" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 34115 } } peers { permanent_uuid: "8e8fcf7e414b4304a18a79967f8a53ba" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 44931 } } }
I20260809 22:52:32.911945 19312 raft_consensus.cc:729] T 84ffa14bf2a34a0da42bb1d66764ba4a P 0c75510608d24ff78855de400475c524 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0c75510608d24ff78855de400475c524, State: Initialized, Role: FOLLOWER
I20260809 22:52:32.911979 19324 tablet_replica.cc:442] T 84ffa14bf2a34a0da42bb1d66764ba4a P 8e8fcf7e414b4304a18a79967f8a53ba: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 22:52:32.912168 19312 consensus_queue.cc:260] T 84ffa14bf2a34a0da42bb1d66764ba4a P 0c75510608d24ff78855de400475c524 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa95fcf2448c4166902c56d01332630c" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 35739 } } peers { permanent_uuid: "0c75510608d24ff78855de400475c524" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 34115 } } peers { permanent_uuid: "8e8fcf7e414b4304a18a79967f8a53ba" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 44931 } }
I20260809 22:52:32.912760 19312 ts_tablet_manager.cc:1428] T 84ffa14bf2a34a0da42bb1d66764ba4a P 0c75510608d24ff78855de400475c524: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260809 22:52:32.912600 19326 tablet_replica.cc:439] T 84ffa14bf2a34a0da42bb1d66764ba4a P 0c75510608d24ff78855de400475c524: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa95fcf2448c4166902c56d01332630c" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 35739 } } peers { permanent_uuid: "0c75510608d24ff78855de400475c524" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 34115 } } peers { permanent_uuid: "8e8fcf7e414b4304a18a79967f8a53ba" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 44931 } } }
I20260809 22:52:32.912705 19311 raft_consensus.cc:348] T 84ffa14bf2a34a0da42bb1d66764ba4a P fa95fcf2448c4166902c56d01332630c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa95fcf2448c4166902c56d01332630c" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 35739 } } peers { permanent_uuid: "0c75510608d24ff78855de400475c524" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 34115 } } peers { permanent_uuid: "8e8fcf7e414b4304a18a79967f8a53ba" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 44931 } }
I20260809 22:52:32.912932 19326 tablet_replica.cc:442] T 84ffa14bf2a34a0da42bb1d66764ba4a P 0c75510608d24ff78855de400475c524: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 22:52:32.913010 19311 raft_consensus.cc:374] T 84ffa14bf2a34a0da42bb1d66764ba4a P fa95fcf2448c4166902c56d01332630c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:52:32.913101 19311 raft_consensus.cc:729] T 84ffa14bf2a34a0da42bb1d66764ba4a P fa95fcf2448c4166902c56d01332630c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa95fcf2448c4166902c56d01332630c, State: Initialized, Role: FOLLOWER
I20260809 22:52:32.913309 19311 consensus_queue.cc:260] T 84ffa14bf2a34a0da42bb1d66764ba4a P fa95fcf2448c4166902c56d01332630c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa95fcf2448c4166902c56d01332630c" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 35739 } } peers { permanent_uuid: "0c75510608d24ff78855de400475c524" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 34115 } } peers { permanent_uuid: "8e8fcf7e414b4304a18a79967f8a53ba" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 44931 } }
I20260809 22:52:32.913911 19311 ts_tablet_manager.cc:1428] T 84ffa14bf2a34a0da42bb1d66764ba4a P fa95fcf2448c4166902c56d01332630c: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20260809 22:52:32.913822 19327 tablet_replica.cc:439] T 84ffa14bf2a34a0da42bb1d66764ba4a P fa95fcf2448c4166902c56d01332630c: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa95fcf2448c4166902c56d01332630c" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 35739 } } peers { permanent_uuid: "0c75510608d24ff78855de400475c524" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 34115 } } peers { permanent_uuid: "8e8fcf7e414b4304a18a79967f8a53ba" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 44931 } } }
I20260809 22:52:32.914233 19327 tablet_replica.cc:442] T 84ffa14bf2a34a0da42bb1d66764ba4a P fa95fcf2448c4166902c56d01332630c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 22:52:32.948478 19326 raft_consensus.cc:482] T 84ffa14bf2a34a0da42bb1d66764ba4a P 0c75510608d24ff78855de400475c524 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 22:52:32.948689 19326 raft_consensus.cc:504] T 84ffa14bf2a34a0da42bb1d66764ba4a P 0c75510608d24ff78855de400475c524 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa95fcf2448c4166902c56d01332630c" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 35739 } } peers { permanent_uuid: "0c75510608d24ff78855de400475c524" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 34115 } } peers { permanent_uuid: "8e8fcf7e414b4304a18a79967f8a53ba" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 44931 } }
I20260809 22:52:32.949379 19326 leader_election.cc:288] T 84ffa14bf2a34a0da42bb1d66764ba4a P 0c75510608d24ff78855de400475c524 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fa95fcf2448c4166902c56d01332630c (127.15.113.129:35739), 8e8fcf7e414b4304a18a79967f8a53ba (127.15.113.131:44931)
I20260809 22:52:32.949903 19205 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "84ffa14bf2a34a0da42bb1d66764ba4a" candidate_uuid: "0c75510608d24ff78855de400475c524" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e8fcf7e414b4304a18a79967f8a53ba" is_pre_election: true
I20260809 22:52:32.949910 18767 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "84ffa14bf2a34a0da42bb1d66764ba4a" candidate_uuid: "0c75510608d24ff78855de400475c524" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa95fcf2448c4166902c56d01332630c" is_pre_election: true
I20260809 22:52:32.950134 19205 raft_consensus.cc:2454] T 84ffa14bf2a34a0da42bb1d66764ba4a P 8e8fcf7e414b4304a18a79967f8a53ba [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0c75510608d24ff78855de400475c524 in term 0.
I20260809 22:52:32.950158 18767 raft_consensus.cc:2454] T 84ffa14bf2a34a0da42bb1d66764ba4a P fa95fcf2448c4166902c56d01332630c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0c75510608d24ff78855de400475c524 in term 0.
I20260809 22:52:32.950551 18863 leader_election.cc:302] T 84ffa14bf2a34a0da42bb1d66764ba4a P 0c75510608d24ff78855de400475c524 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 0c75510608d24ff78855de400475c524, 8e8fcf7e414b4304a18a79967f8a53ba, fa95fcf2448c4166902c56d01332630c; no voters:
I20260809 22:52:32.950855 19326 raft_consensus.cc:2781] T 84ffa14bf2a34a0da42bb1d66764ba4a P 0c75510608d24ff78855de400475c524 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 22:52:32.950963 19326 raft_consensus.cc:482] T 84ffa14bf2a34a0da42bb1d66764ba4a P 0c75510608d24ff78855de400475c524 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 22:52:32.951068 19326 raft_consensus.cc:3037] T 84ffa14bf2a34a0da42bb1d66764ba4a P 0c75510608d24ff78855de400475c524 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:52:32.953079 19326 raft_consensus.cc:504] T 84ffa14bf2a34a0da42bb1d66764ba4a P 0c75510608d24ff78855de400475c524 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa95fcf2448c4166902c56d01332630c" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 35739 } } peers { permanent_uuid: "0c75510608d24ff78855de400475c524" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 34115 } } peers { permanent_uuid: "8e8fcf7e414b4304a18a79967f8a53ba" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 44931 } }
I20260809 22:52:32.953564 19326 leader_election.cc:288] T 84ffa14bf2a34a0da42bb1d66764ba4a P 0c75510608d24ff78855de400475c524 [CANDIDATE]: Term 1 election: Requested vote from peers fa95fcf2448c4166902c56d01332630c (127.15.113.129:35739), 8e8fcf7e414b4304a18a79967f8a53ba (127.15.113.131:44931)
I20260809 22:52:32.953912 18767 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "84ffa14bf2a34a0da42bb1d66764ba4a" candidate_uuid: "0c75510608d24ff78855de400475c524" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa95fcf2448c4166902c56d01332630c"
I20260809 22:52:32.953992 19205 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "84ffa14bf2a34a0da42bb1d66764ba4a" candidate_uuid: "0c75510608d24ff78855de400475c524" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e8fcf7e414b4304a18a79967f8a53ba"
I20260809 22:52:32.954092 18767 raft_consensus.cc:3037] T 84ffa14bf2a34a0da42bb1d66764ba4a P fa95fcf2448c4166902c56d01332630c [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:52:32.954205 19205 raft_consensus.cc:3037] T 84ffa14bf2a34a0da42bb1d66764ba4a P 8e8fcf7e414b4304a18a79967f8a53ba [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:52:32.955968 18767 raft_consensus.cc:2454] T 84ffa14bf2a34a0da42bb1d66764ba4a P fa95fcf2448c4166902c56d01332630c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0c75510608d24ff78855de400475c524 in term 1.
I20260809 22:52:32.956141 19205 raft_consensus.cc:2454] T 84ffa14bf2a34a0da42bb1d66764ba4a P 8e8fcf7e414b4304a18a79967f8a53ba [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0c75510608d24ff78855de400475c524 in term 1.
I20260809 22:52:32.956305 18867 leader_election.cc:302] T 84ffa14bf2a34a0da42bb1d66764ba4a P 0c75510608d24ff78855de400475c524 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0c75510608d24ff78855de400475c524, fa95fcf2448c4166902c56d01332630c; no voters:
I20260809 22:52:32.956544 19326 raft_consensus.cc:2781] T 84ffa14bf2a34a0da42bb1d66764ba4a P 0c75510608d24ff78855de400475c524 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 22:52:32.956700 19326 raft_consensus.cc:686] T 84ffa14bf2a34a0da42bb1d66764ba4a P 0c75510608d24ff78855de400475c524 [term 1 LEADER]: Becoming Leader. State: Replica: 0c75510608d24ff78855de400475c524, State: Running, Role: LEADER
I20260809 22:52:32.956962 19326 consensus_queue.cc:237] T 84ffa14bf2a34a0da42bb1d66764ba4a P 0c75510608d24ff78855de400475c524 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa95fcf2448c4166902c56d01332630c" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 35739 } } peers { permanent_uuid: "0c75510608d24ff78855de400475c524" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 34115 } } peers { permanent_uuid: "8e8fcf7e414b4304a18a79967f8a53ba" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 44931 } }
I20260809 22:52:32.958607 19330 tablet_replica.cc:439] T 84ffa14bf2a34a0da42bb1d66764ba4a P 0c75510608d24ff78855de400475c524: TxnStatusTablet state changed. Reason: New leader 0c75510608d24ff78855de400475c524. Latest consensus state: current_term: 1 leader_uuid: "0c75510608d24ff78855de400475c524" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa95fcf2448c4166902c56d01332630c" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 35739 } } peers { permanent_uuid: "0c75510608d24ff78855de400475c524" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 34115 } } peers { permanent_uuid: "8e8fcf7e414b4304a18a79967f8a53ba" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 44931 } } }
I20260809 22:52:32.958958 19330 tablet_replica.cc:442] T 84ffa14bf2a34a0da42bb1d66764ba4a P 0c75510608d24ff78855de400475c524: This TxnStatusTablet replica's current role is: LEADER
I20260809 22:52:32.959692 19350 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260809 22:52:32.960006 19350 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260809 22:52:32.960072 18513 catalog_manager.cc:5495] T 84ffa14bf2a34a0da42bb1d66764ba4a P 0c75510608d24ff78855de400475c524 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0c75510608d24ff78855de400475c524 (127.15.113.130). New cstate: current_term: 1 leader_uuid: "0c75510608d24ff78855de400475c524" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa95fcf2448c4166902c56d01332630c" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 35739 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0c75510608d24ff78855de400475c524" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 34115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8e8fcf7e414b4304a18a79967f8a53ba" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 44931 } health_report { overall_health: UNKNOWN } } }
I20260809 22:52:32.990018 19340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:52:32.984019 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.130:34115 (local address 127.0.0.1:38388)
0809 22:52:32.984262 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0809 22:52:32.984282 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 22:52:32.984401 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:52:32.984842 (+ 441us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:52:32.984846 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:52:32.984855 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 22:52:32.985067 (+ 212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:52:32.985080 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:52:32.986515 (+ 1435us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:52:32.986519 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:52:32.987489 (+ 970us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:52:32.987500 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:52:32.988426 (+ 926us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:52:32.988430 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:52:32.988572 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:52:32.988623 (+ 51us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 22:52:32.989612 (+ 989us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 22:52:32.989627 (+ 15us) client_negotiation.cc:769] Sending connection context
0809 22:52:32.989678 (+ 51us) client_negotiation.cc:240] Negotiation successful
0809 22:52:32.989788 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":50}
I20260809 22:52:32.990075 19303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:52:32.984189 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38388 (local address 127.15.113.130:34115)
0809 22:52:32.984404 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0809 22:52:32.984410 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:52:32.984434 (+ 24us) server_negotiation.cc:407] Connection header received
0809 22:52:32.984572 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:52:32.984578 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:52:32.984687 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:52:32.984842 (+ 155us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 22:52:32.985228 (+ 386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:52:32.986386 (+ 1158us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:52:32.987621 (+ 1235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:52:32.988294 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:52:32.988417 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:52:32.988817 (+ 400us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 22:52:32.989519 (+ 702us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 22:52:32.989670 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0809 22:52:32.989718 (+ 48us) server_negotiation.cc:299] Negotiation successful
0809 22:52:32.989797 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260809 22:52:32.998811 19205 raft_consensus.cc:1261] T 84ffa14bf2a34a0da42bb1d66764ba4a P 8e8fcf7e414b4304a18a79967f8a53ba [term 1 FOLLOWER]: Refusing update from remote peer 0c75510608d24ff78855de400475c524: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 22:52:32.998950 18767 raft_consensus.cc:1261] T 84ffa14bf2a34a0da42bb1d66764ba4a P fa95fcf2448c4166902c56d01332630c [term 1 FOLLOWER]: Refusing update from remote peer 0c75510608d24ff78855de400475c524: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 22:52:32.999871 19330 consensus_queue.cc:1035] T 84ffa14bf2a34a0da42bb1d66764ba4a P 0c75510608d24ff78855de400475c524 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e8fcf7e414b4304a18a79967f8a53ba" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 44931 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 22:52:33.000191 19326 consensus_queue.cc:1035] T 84ffa14bf2a34a0da42bb1d66764ba4a P 0c75510608d24ff78855de400475c524 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fa95fcf2448c4166902c56d01332630c" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 35739 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 22:52:33.017479 19327 tablet_replica.cc:439] T 84ffa14bf2a34a0da42bb1d66764ba4a P fa95fcf2448c4166902c56d01332630c: TxnStatusTablet state changed. Reason: New leader 0c75510608d24ff78855de400475c524. Latest consensus state: current_term: 1 leader_uuid: "0c75510608d24ff78855de400475c524" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa95fcf2448c4166902c56d01332630c" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 35739 } } peers { permanent_uuid: "0c75510608d24ff78855de400475c524" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 34115 } } peers { permanent_uuid: "8e8fcf7e414b4304a18a79967f8a53ba" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 44931 } } }
I20260809 22:52:33.017773 19327 tablet_replica.cc:442] T 84ffa14bf2a34a0da42bb1d66764ba4a P fa95fcf2448c4166902c56d01332630c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 22:52:33.017849 19324 tablet_replica.cc:439] T 84ffa14bf2a34a0da42bb1d66764ba4a P 8e8fcf7e414b4304a18a79967f8a53ba: TxnStatusTablet state changed. Reason: New leader 0c75510608d24ff78855de400475c524. Latest consensus state: current_term: 1 leader_uuid: "0c75510608d24ff78855de400475c524" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa95fcf2448c4166902c56d01332630c" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 35739 } } peers { permanent_uuid: "0c75510608d24ff78855de400475c524" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 34115 } } peers { permanent_uuid: "8e8fcf7e414b4304a18a79967f8a53ba" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 44931 } } }
I20260809 22:52:33.018144 19324 tablet_replica.cc:442] T 84ffa14bf2a34a0da42bb1d66764ba4a P 8e8fcf7e414b4304a18a79967f8a53ba: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 22:52:33.021710 19355 mvcc.cc:205] Tried to move back new op lower bound from 7316750143472693248 to 7316750143313833984. Current Snapshot: MvccSnapshot[applied={T|T < 7316750143472693248}]
I20260809 22:52:33.022755 19326 tablet_replica.cc:439] T 84ffa14bf2a34a0da42bb1d66764ba4a P 0c75510608d24ff78855de400475c524: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "0c75510608d24ff78855de400475c524" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa95fcf2448c4166902c56d01332630c" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 35739 } } peers { permanent_uuid: "0c75510608d24ff78855de400475c524" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 34115 } } peers { permanent_uuid: "8e8fcf7e414b4304a18a79967f8a53ba" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 44931 } } }
I20260809 22:52:33.023046 19326 tablet_replica.cc:442] T 84ffa14bf2a34a0da42bb1d66764ba4a P 0c75510608d24ff78855de400475c524: This TxnStatusTablet replica's current role is: LEADER
I20260809 22:52:33.023301 19326 tablet_replica.cc:439] T 84ffa14bf2a34a0da42bb1d66764ba4a P 0c75510608d24ff78855de400475c524: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "0c75510608d24ff78855de400475c524" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa95fcf2448c4166902c56d01332630c" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 35739 } } peers { permanent_uuid: "0c75510608d24ff78855de400475c524" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 34115 } } peers { permanent_uuid: "8e8fcf7e414b4304a18a79967f8a53ba" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 44931 } } }
I20260809 22:52:33.023559 19326 tablet_replica.cc:442] T 84ffa14bf2a34a0da42bb1d66764ba4a P 0c75510608d24ff78855de400475c524: This TxnStatusTablet replica's current role is: LEADER
I20260809 22:52:33.023679 19357 mvcc.cc:205] Tried to move back new op lower bound from 7316750143472693248 to 7316750143313833984. Current Snapshot: MvccSnapshot[applied={T|T < 7316750143472693248}]
I20260809 22:52:33.036850 19327 tablet_replica.cc:439] T 84ffa14bf2a34a0da42bb1d66764ba4a P fa95fcf2448c4166902c56d01332630c: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "0c75510608d24ff78855de400475c524" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa95fcf2448c4166902c56d01332630c" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 35739 } } peers { permanent_uuid: "0c75510608d24ff78855de400475c524" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 34115 } } peers { permanent_uuid: "8e8fcf7e414b4304a18a79967f8a53ba" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 44931 } } }
I20260809 22:52:33.037133 19327 tablet_replica.cc:442] T 84ffa14bf2a34a0da42bb1d66764ba4a P fa95fcf2448c4166902c56d01332630c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 22:52:33.038702 19324 tablet_replica.cc:439] T 84ffa14bf2a34a0da42bb1d66764ba4a P 8e8fcf7e414b4304a18a79967f8a53ba: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "0c75510608d24ff78855de400475c524" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa95fcf2448c4166902c56d01332630c" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 35739 } } peers { permanent_uuid: "0c75510608d24ff78855de400475c524" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 34115 } } peers { permanent_uuid: "8e8fcf7e414b4304a18a79967f8a53ba" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 44931 } } }
I20260809 22:52:33.038941 19324 tablet_replica.cc:442] T 84ffa14bf2a34a0da42bb1d66764ba4a P 8e8fcf7e414b4304a18a79967f8a53ba: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 22:52:33.050777 19303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:52:33.045732 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38392 (local address 127.15.113.130:34115)
0809 22:52:33.045901 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0809 22:52:33.045907 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:52:33.046005 (+ 98us) server_negotiation.cc:407] Connection header received
0809 22:52:33.046219 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:52:33.046223 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:52:33.046285 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:52:33.046360 (+ 75us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 22:52:33.046735 (+ 375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:52:33.047481 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:52:33.048820 (+ 1339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:52:33.049549 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:52:33.049707 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:52:33.050049 (+ 342us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 22:52:33.050208 (+ 159us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 22:52:33.050297 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0809 22:52:33.050378 (+ 81us) server_negotiation.cc:299] Negotiation successful
0809 22:52:33.050459 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260809 22:52:33.067988 19303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:52:33.061685 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.113.130:40905 (local address 127.15.113.130:34115)
0809 22:52:33.061839 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0809 22:52:33.061843 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 22:52:33.062273 (+ 430us) server_negotiation.cc:407] Connection header received
0809 22:52:33.062413 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:52:33.062418 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:52:33.062501 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:52:33.062594 (+ 93us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 22:52:33.063055 (+ 461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:52:33.064233 (+ 1178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:52:33.066344 (+ 2111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:52:33.066760 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:52:33.066850 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:52:33.067377 (+ 527us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 22:52:33.067462 (+ 85us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 22:52:33.067525 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0809 22:52:33.067677 (+ 152us) server_negotiation.cc:299] Negotiation successful
0809 22:52:33.067731 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260809 22:52:33.068296 19377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:52:33.061578 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.130:34115 (local address 127.15.113.130:40905)
0809 22:52:33.062165 (+ 587us) negotiation.cc:107] Waiting for socket to connect
0809 22:52:33.062193 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0809 22:52:33.062301 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:52:33.062604 (+ 303us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:52:33.062607 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:52:33.062615 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 22:52:33.062915 (+ 300us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:52:33.062923 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:52:33.064387 (+ 1464us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:52:33.064393 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:52:33.066213 (+ 1820us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:52:33.066229 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:52:33.066932 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:52:33.066939 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:52:33.067122 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:52:33.067189 (+ 67us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 22:52:33.067553 (+ 364us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 22:52:33.067571 (+ 18us) client_negotiation.cc:769] Sending connection context
0809 22:52:33.067665 (+ 94us) client_negotiation.cc:240] Negotiation successful
0809 22:52:33.067807 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":468,"thread_start_us":358,"threads_started":1}
I20260809 22:52:33.076753 19205 raft_consensus.cc:1261] T 53dcd318d9c24d79acb67622a42e83ec P 8e8fcf7e414b4304a18a79967f8a53ba [term 1 FOLLOWER]: Refusing update from remote peer 0c75510608d24ff78855de400475c524: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 22:52:33.076861 18767 raft_consensus.cc:1261] T 53dcd318d9c24d79acb67622a42e83ec P fa95fcf2448c4166902c56d01332630c [term 1 FOLLOWER]: Refusing update from remote peer 0c75510608d24ff78855de400475c524: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 22:52:33.077381 19365 consensus_queue.cc:1035] T 53dcd318d9c24d79acb67622a42e83ec P 0c75510608d24ff78855de400475c524 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e8fcf7e414b4304a18a79967f8a53ba" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 44931 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 22:52:33.077900 19365 consensus_queue.cc:1035] T 53dcd318d9c24d79acb67622a42e83ec P 0c75510608d24ff78855de400475c524 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fa95fcf2448c4166902c56d01332630c" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 35739 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 22:52:33.102941 19356 mvcc.cc:205] Tried to move back new op lower bound from 7316750143797977088 to 7316750142809706496. Current Snapshot: MvccSnapshot[applied={T|T < 7316750143797977088}]
I20260809 22:52:33.176460 18511 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60638:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 22:52:33.176770 18511 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60638:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 22:52:33.180550 18511 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 72c0d959ee5c41298a2ea15ccc17437f: Sending DeleteTablet for 3 replicas of tablet 53dcd318d9c24d79acb67622a42e83ec
I20260809 22:52:33.181536 18947 tablet_service.cc:1514] Processing DeleteTablet for tablet 53dcd318d9c24d79acb67622a42e83ec with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:52:33 UTC) from {username='kudu', principal='kudu/127.15.113.190@KRBTEST.COM'} at 127.0.0.1:38374
I20260809 22:52:33.181730 19161 tablet_service.cc:1514] Processing DeleteTablet for tablet 53dcd318d9c24d79acb67622a42e83ec with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:52:33 UTC) from {username='kudu', principal='kudu/127.15.113.190@KRBTEST.COM'} at 127.0.0.1:48732
I20260809 22:52:33.181731 18732 tablet_service.cc:1514] Processing DeleteTablet for tablet 53dcd318d9c24d79acb67622a42e83ec with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:52:33 UTC) from {username='kudu', principal='kudu/127.15.113.190@KRBTEST.COM'} at 127.0.0.1:34790
I20260809 22:52:33.184049 18511 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60638:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 22:52:33.185335 18511 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 22:52:33.194303 19395 tablet_replica.cc:330] T 53dcd318d9c24d79acb67622a42e83ec P 8e8fcf7e414b4304a18a79967f8a53ba: stopping tablet replica
I20260809 22:52:33.194482 19394 tablet_replica.cc:330] T 53dcd318d9c24d79acb67622a42e83ec P fa95fcf2448c4166902c56d01332630c: stopping tablet replica
I20260809 22:52:33.194761 18729 tablet_service.cc:1467] Processing CreateTablet for tablet 61b0b10f318a4d169acd393b1b331de4 (DEFAULT_TABLE table=test-table [id=ebeb248942b34470809e13a26df1133a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:52:33.195068 19394 raft_consensus.cc:2229] T 53dcd318d9c24d79acb67622a42e83ec P fa95fcf2448c4166902c56d01332630c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 22:52:33.195243 19395 raft_consensus.cc:2229] T 53dcd318d9c24d79acb67622a42e83ec P 8e8fcf7e414b4304a18a79967f8a53ba [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 22:52:33.195355 18729 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 61b0b10f318a4d169acd393b1b331de4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:52:33.195470 19394 raft_consensus.cc:2258] T 53dcd318d9c24d79acb67622a42e83ec P fa95fcf2448c4166902c56d01332630c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 22:52:33.195587 19395 raft_consensus.cc:2258] T 53dcd318d9c24d79acb67622a42e83ec P 8e8fcf7e414b4304a18a79967f8a53ba [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 22:52:33.197232 19392 tablet_replica.cc:330] T 53dcd318d9c24d79acb67622a42e83ec P 0c75510608d24ff78855de400475c524: stopping tablet replica
I20260809 22:52:33.197635 19394 ts_tablet_manager.cc:1905] T 53dcd318d9c24d79acb67622a42e83ec P fa95fcf2448c4166902c56d01332630c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 22:52:33.197665 19392 raft_consensus.cc:2229] T 53dcd318d9c24d79acb67622a42e83ec P 0c75510608d24ff78855de400475c524 [term 1 LEADER]: Raft consensus shutting down.
I20260809 22:52:33.198338 19392 raft_consensus.cc:2258] T 53dcd318d9c24d79acb67622a42e83ec P 0c75510608d24ff78855de400475c524 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 22:52:33.199249 18945 tablet_service.cc:1467] Processing CreateTablet for tablet 61b0b10f318a4d169acd393b1b331de4 (DEFAULT_TABLE table=test-table [id=ebeb248942b34470809e13a26df1133a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:52:33.199683 18945 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 61b0b10f318a4d169acd393b1b331de4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:52:33.204137 19395 ts_tablet_manager.cc:1905] T 53dcd318d9c24d79acb67622a42e83ec P 8e8fcf7e414b4304a18a79967f8a53ba: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 22:52:33.204629 19311 tablet_bootstrap.cc:492] T 61b0b10f318a4d169acd393b1b331de4 P fa95fcf2448c4166902c56d01332630c: Bootstrap starting.
I20260809 22:52:33.205024 19394 ts_tablet_manager.cc:1918] T 53dcd318d9c24d79acb67622a42e83ec P fa95fcf2448c4166902c56d01332630c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 22:52:33.204991 19161 tablet_service.cc:1467] Processing CreateTablet for tablet 61b0b10f318a4d169acd393b1b331de4 (DEFAULT_TABLE table=test-table [id=ebeb248942b34470809e13a26df1133a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:52:33.205184 19394 log.cc:1196] T 53dcd318d9c24d79acb67622a42e83ec P fa95fcf2448c4166902c56d01332630c: Deleting WAL directory at /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315935617919-15814-0/minicluster-data/ts-0/wal/wals/53dcd318d9c24d79acb67622a42e83ec
I20260809 22:52:33.205394 19161 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 61b0b10f318a4d169acd393b1b331de4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:52:33.205775 19394 ts_tablet_manager.cc:1939] T 53dcd318d9c24d79acb67622a42e83ec P fa95fcf2448c4166902c56d01332630c: Deleting consensus metadata
I20260809 22:52:33.206354 19311 tablet_bootstrap.cc:654] T 61b0b10f318a4d169acd393b1b331de4 P fa95fcf2448c4166902c56d01332630c: Neither blocks nor log segments found. Creating new log.
I20260809 22:52:33.207482 18491 catalog_manager.cc:4841] TS fa95fcf2448c4166902c56d01332630c (127.15.113.129:35739): tablet 53dcd318d9c24d79acb67622a42e83ec (table test-table [id=d488e6510c1e466eaab14daa65952b12]) successfully deleted
I20260809 22:52:33.210476 19312 tablet_bootstrap.cc:492] T 61b0b10f318a4d169acd393b1b331de4 P 0c75510608d24ff78855de400475c524: Bootstrap starting.
I20260809 22:52:33.211477 19395 ts_tablet_manager.cc:1918] T 53dcd318d9c24d79acb67622a42e83ec P 8e8fcf7e414b4304a18a79967f8a53ba: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 22:52:33.211612 19395 log.cc:1196] T 53dcd318d9c24d79acb67622a42e83ec P 8e8fcf7e414b4304a18a79967f8a53ba: Deleting WAL directory at /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315935617919-15814-0/minicluster-data/ts-2/wal/wals/53dcd318d9c24d79acb67622a42e83ec
I20260809 22:52:33.212101 19395 ts_tablet_manager.cc:1939] T 53dcd318d9c24d79acb67622a42e83ec P 8e8fcf7e414b4304a18a79967f8a53ba: Deleting consensus metadata
I20260809 22:52:33.212146 19312 tablet_bootstrap.cc:654] T 61b0b10f318a4d169acd393b1b331de4 P 0c75510608d24ff78855de400475c524: Neither blocks nor log segments found. Creating new log.
I20260809 22:52:33.213752 18487 catalog_manager.cc:4841] TS 8e8fcf7e414b4304a18a79967f8a53ba (127.15.113.131:44931): tablet 53dcd318d9c24d79acb67622a42e83ec (table test-table [id=d488e6510c1e466eaab14daa65952b12]) successfully deleted
I20260809 22:52:33.215040 19310 tablet_bootstrap.cc:492] T 61b0b10f318a4d169acd393b1b331de4 P 8e8fcf7e414b4304a18a79967f8a53ba: Bootstrap starting.
I20260809 22:52:33.216331 19392 ts_tablet_manager.cc:1905] T 53dcd318d9c24d79acb67622a42e83ec P 0c75510608d24ff78855de400475c524: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 22:52:33.216543 19310 tablet_bootstrap.cc:654] T 61b0b10f318a4d169acd393b1b331de4 P 8e8fcf7e414b4304a18a79967f8a53ba: Neither blocks nor log segments found. Creating new log.
I20260809 22:52:33.218223 19311 tablet_bootstrap.cc:492] T 61b0b10f318a4d169acd393b1b331de4 P fa95fcf2448c4166902c56d01332630c: No bootstrap required, opened a new log
I20260809 22:52:33.219435 19311 ts_tablet_manager.cc:1397] T 61b0b10f318a4d169acd393b1b331de4 P fa95fcf2448c4166902c56d01332630c: Time spent bootstrapping tablet: real 0.015s user 0.007s sys 0.000s
I20260809 22:52:33.220237 19311 raft_consensus.cc:348] T 61b0b10f318a4d169acd393b1b331de4 P fa95fcf2448c4166902c56d01332630c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa95fcf2448c4166902c56d01332630c" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 35739 } } peers { permanent_uuid: "0c75510608d24ff78855de400475c524" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 34115 } } peers { permanent_uuid: "8e8fcf7e414b4304a18a79967f8a53ba" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 44931 } }
I20260809 22:52:33.220623 19311 raft_consensus.cc:374] T 61b0b10f318a4d169acd393b1b331de4 P fa95fcf2448c4166902c56d01332630c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:52:33.220744 19311 raft_consensus.cc:729] T 61b0b10f318a4d169acd393b1b331de4 P fa95fcf2448c4166902c56d01332630c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa95fcf2448c4166902c56d01332630c, State: Initialized, Role: FOLLOWER
I20260809 22:52:33.221112 19311 consensus_queue.cc:260] T 61b0b10f318a4d169acd393b1b331de4 P fa95fcf2448c4166902c56d01332630c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa95fcf2448c4166902c56d01332630c" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 35739 } } peers { permanent_uuid: "0c75510608d24ff78855de400475c524" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 34115 } } peers { permanent_uuid: "8e8fcf7e414b4304a18a79967f8a53ba" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 44931 } }
I20260809 22:52:33.221836 19311 ts_tablet_manager.cc:1428] T 61b0b10f318a4d169acd393b1b331de4 P fa95fcf2448c4166902c56d01332630c: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260809 22:52:33.225700 19310 tablet_bootstrap.cc:492] T 61b0b10f318a4d169acd393b1b331de4 P 8e8fcf7e414b4304a18a79967f8a53ba: No bootstrap required, opened a new log
I20260809 22:52:33.225872 19310 ts_tablet_manager.cc:1397] T 61b0b10f318a4d169acd393b1b331de4 P 8e8fcf7e414b4304a18a79967f8a53ba: Time spent bootstrapping tablet: real 0.011s user 0.007s sys 0.000s
I20260809 22:52:33.226718 19312 tablet_bootstrap.cc:492] T 61b0b10f318a4d169acd393b1b331de4 P 0c75510608d24ff78855de400475c524: No bootstrap required, opened a new log
I20260809 22:52:33.226593 19310 raft_consensus.cc:348] T 61b0b10f318a4d169acd393b1b331de4 P 8e8fcf7e414b4304a18a79967f8a53ba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa95fcf2448c4166902c56d01332630c" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 35739 } } peers { permanent_uuid: "0c75510608d24ff78855de400475c524" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 34115 } } peers { permanent_uuid: "8e8fcf7e414b4304a18a79967f8a53ba" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 44931 } }
I20260809 22:52:33.226824 19310 raft_consensus.cc:374] T 61b0b10f318a4d169acd393b1b331de4 P 8e8fcf7e414b4304a18a79967f8a53ba [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:52:33.226969 19312 ts_tablet_manager.cc:1397] T 61b0b10f318a4d169acd393b1b331de4 P 0c75510608d24ff78855de400475c524: Time spent bootstrapping tablet: real 0.017s user 0.012s sys 0.003s
I20260809 22:52:33.226959 19310 raft_consensus.cc:729] T 61b0b10f318a4d169acd393b1b331de4 P 8e8fcf7e414b4304a18a79967f8a53ba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8e8fcf7e414b4304a18a79967f8a53ba, State: Initialized, Role: FOLLOWER
I20260809 22:52:33.227250 19310 consensus_queue.cc:260] T 61b0b10f318a4d169acd393b1b331de4 P 8e8fcf7e414b4304a18a79967f8a53ba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa95fcf2448c4166902c56d01332630c" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 35739 } } peers { permanent_uuid: "0c75510608d24ff78855de400475c524" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 34115 } } peers { permanent_uuid: "8e8fcf7e414b4304a18a79967f8a53ba" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 44931 } }
I20260809 22:52:33.227826 19312 raft_consensus.cc:348] T 61b0b10f318a4d169acd393b1b331de4 P 0c75510608d24ff78855de400475c524 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa95fcf2448c4166902c56d01332630c" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 35739 } } peers { permanent_uuid: "0c75510608d24ff78855de400475c524" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 34115 } } peers { permanent_uuid: "8e8fcf7e414b4304a18a79967f8a53ba" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 44931 } }
I20260809 22:52:33.228135 19312 raft_consensus.cc:374] T 61b0b10f318a4d169acd393b1b331de4 P 0c75510608d24ff78855de400475c524 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:52:33.228273 19312 raft_consensus.cc:729] T 61b0b10f318a4d169acd393b1b331de4 P 0c75510608d24ff78855de400475c524 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0c75510608d24ff78855de400475c524, State: Initialized, Role: FOLLOWER
I20260809 22:52:33.228606 19312 consensus_queue.cc:260] T 61b0b10f318a4d169acd393b1b331de4 P 0c75510608d24ff78855de400475c524 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa95fcf2448c4166902c56d01332630c" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 35739 } } peers { permanent_uuid: "0c75510608d24ff78855de400475c524" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 34115 } } peers { permanent_uuid: "8e8fcf7e414b4304a18a79967f8a53ba" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 44931 } }
I20260809 22:52:33.230124 19310 ts_tablet_manager.cc:1428] T 61b0b10f318a4d169acd393b1b331de4 P 8e8fcf7e414b4304a18a79967f8a53ba: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260809 22:52:33.231247 19392 ts_tablet_manager.cc:1918] T 53dcd318d9c24d79acb67622a42e83ec P 0c75510608d24ff78855de400475c524: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 22:52:33.231380 19312 ts_tablet_manager.cc:1428] T 61b0b10f318a4d169acd393b1b331de4 P 0c75510608d24ff78855de400475c524: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260809 22:52:33.231503 19392 log.cc:1196] T 53dcd318d9c24d79acb67622a42e83ec P 0c75510608d24ff78855de400475c524: Deleting WAL directory at /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315935617919-15814-0/minicluster-data/ts-1/wal/wals/53dcd318d9c24d79acb67622a42e83ec
I20260809 22:52:33.232229 19392 ts_tablet_manager.cc:1939] T 53dcd318d9c24d79acb67622a42e83ec P 0c75510608d24ff78855de400475c524: Deleting consensus metadata
I20260809 22:52:33.233929 18491 catalog_manager.cc:4841] TS 0c75510608d24ff78855de400475c524 (127.15.113.130:34115): tablet 53dcd318d9c24d79acb67622a42e83ec (table test-table [id=d488e6510c1e466eaab14daa65952b12]) successfully deleted
I20260809 22:52:33.375834 19324 raft_consensus.cc:482] T 61b0b10f318a4d169acd393b1b331de4 P 8e8fcf7e414b4304a18a79967f8a53ba [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 22:52:33.376080 19324 raft_consensus.cc:504] T 61b0b10f318a4d169acd393b1b331de4 P 8e8fcf7e414b4304a18a79967f8a53ba [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa95fcf2448c4166902c56d01332630c" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 35739 } } peers { permanent_uuid: "0c75510608d24ff78855de400475c524" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 34115 } } peers { permanent_uuid: "8e8fcf7e414b4304a18a79967f8a53ba" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 44931 } }
I20260809 22:52:33.377945 19324 leader_election.cc:288] T 61b0b10f318a4d169acd393b1b331de4 P 8e8fcf7e414b4304a18a79967f8a53ba [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fa95fcf2448c4166902c56d01332630c (127.15.113.129:35739), 0c75510608d24ff78855de400475c524 (127.15.113.130:34115)
I20260809 22:52:33.388343 19406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:52:33.378146 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.129:35739 (local address 127.15.113.131:52007)
0809 22:52:33.381288 (+ 3142us) negotiation.cc:107] Waiting for socket to connect
0809 22:52:33.381309 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0809 22:52:33.381535 (+ 226us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:52:33.382322 (+ 787us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:52:33.382328 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:52:33.382356 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:52:33.382604 (+ 248us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:52:33.382616 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:52:33.384225 (+ 1609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:52:33.384229 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:52:33.386374 (+ 2145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:52:33.386389 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:52:33.387496 (+ 1107us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:52:33.387502 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:52:33.387674 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:52:33.387702 (+ 28us) client_negotiation.cc:769] Sending connection context
0809 22:52:33.387795 (+ 93us) client_negotiation.cc:240] Negotiation successful
0809 22:52:33.387914 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3010,"mutex_wait_us":26,"thread_start_us":2875,"threads_started":1}
I20260809 22:52:33.388638 19407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:52:33.378248 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.113.131:52007 (local address 127.15.113.129:35739)
0809 22:52:33.381962 (+ 3714us) server_negotiation.cc:206] Beginning negotiation
0809 22:52:33.381968 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:52:33.381987 (+ 19us) server_negotiation.cc:407] Connection header received
0809 22:52:33.382057 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:52:33.382062 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:52:33.382134 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:52:33.382767 (+ 633us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:52:33.382951 (+ 184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:52:33.384103 (+ 1152us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:52:33.386574 (+ 2471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:52:33.387340 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:52:33.387495 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:52:33.387572 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0809 22:52:33.388190 (+ 618us) server_negotiation.cc:299] Negotiation successful
0809 22:52:33.388336 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3600,"thread_start_us":3495,"threads_started":1}
I20260809 22:52:33.388873 19303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:52:33.381631 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.113.131:50445 (local address 127.15.113.130:34115)
0809 22:52:33.381834 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0809 22:52:33.381840 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:52:33.382541 (+ 701us) server_negotiation.cc:407] Connection header received
0809 22:52:33.382737 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:52:33.382740 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:52:33.382795 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:52:33.382865 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:52:33.383335 (+ 470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:52:33.384021 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:52:33.386529 (+ 2508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:52:33.387889 (+ 1360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:52:33.388027 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:52:33.388093 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0809 22:52:33.388488 (+ 395us) server_negotiation.cc:299] Negotiation successful
0809 22:52:33.388610 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260809 22:52:33.389405 18766 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "61b0b10f318a4d169acd393b1b331de4" candidate_uuid: "8e8fcf7e414b4304a18a79967f8a53ba" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa95fcf2448c4166902c56d01332630c" is_pre_election: true
I20260809 22:52:33.389534 19409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:52:33.381962 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.130:34115 (local address 127.15.113.131:50445)
0809 22:52:33.382429 (+ 467us) negotiation.cc:107] Waiting for socket to connect
0809 22:52:33.382447 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 22:52:33.382583 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:52:33.382899 (+ 316us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:52:33.382903 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:52:33.382926 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:52:33.383185 (+ 259us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:52:33.383194 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:52:33.384165 (+ 971us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:52:33.384170 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:52:33.386374 (+ 2204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:52:33.386389 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:52:33.388052 (+ 1663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:52:33.388058 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:52:33.388280 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:52:33.388317 (+ 37us) client_negotiation.cc:769] Sending connection context
0809 22:52:33.389071 (+ 754us) client_negotiation.cc:240] Negotiation successful
0809 22:52:33.389201 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":366,"mutex_wait_us":10,"thread_start_us":236,"threads_started":1}
I20260809 22:52:33.389670 18766 raft_consensus.cc:2454] T 61b0b10f318a4d169acd393b1b331de4 P fa95fcf2448c4166902c56d01332630c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e8fcf7e414b4304a18a79967f8a53ba in term 0.
I20260809 22:52:33.390113 19093 leader_election.cc:302] T 61b0b10f318a4d169acd393b1b331de4 P 8e8fcf7e414b4304a18a79967f8a53ba [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8e8fcf7e414b4304a18a79967f8a53ba, fa95fcf2448c4166902c56d01332630c; no voters:
I20260809 22:52:33.390478 19324 raft_consensus.cc:2781] T 61b0b10f318a4d169acd393b1b331de4 P 8e8fcf7e414b4304a18a79967f8a53ba [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 22:52:33.390588 19324 raft_consensus.cc:482] T 61b0b10f318a4d169acd393b1b331de4 P 8e8fcf7e414b4304a18a79967f8a53ba [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 22:52:33.390422 18976 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "61b0b10f318a4d169acd393b1b331de4" candidate_uuid: "8e8fcf7e414b4304a18a79967f8a53ba" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0c75510608d24ff78855de400475c524" is_pre_election: true
I20260809 22:52:33.390692 19324 raft_consensus.cc:3037] T 61b0b10f318a4d169acd393b1b331de4 P 8e8fcf7e414b4304a18a79967f8a53ba [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:52:33.390839 18976 raft_consensus.cc:2454] T 61b0b10f318a4d169acd393b1b331de4 P 0c75510608d24ff78855de400475c524 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e8fcf7e414b4304a18a79967f8a53ba in term 0.
I20260809 22:52:33.393069 19324 raft_consensus.cc:504] T 61b0b10f318a4d169acd393b1b331de4 P 8e8fcf7e414b4304a18a79967f8a53ba [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa95fcf2448c4166902c56d01332630c" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 35739 } } peers { permanent_uuid: "0c75510608d24ff78855de400475c524" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 34115 } } peers { permanent_uuid: "8e8fcf7e414b4304a18a79967f8a53ba" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 44931 } }
I20260809 22:52:33.393657 19324 leader_election.cc:288] T 61b0b10f318a4d169acd393b1b331de4 P 8e8fcf7e414b4304a18a79967f8a53ba [CANDIDATE]: Term 1 election: Requested vote from peers fa95fcf2448c4166902c56d01332630c (127.15.113.129:35739), 0c75510608d24ff78855de400475c524 (127.15.113.130:34115)
I20260809 22:52:33.394057 18766 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "61b0b10f318a4d169acd393b1b331de4" candidate_uuid: "8e8fcf7e414b4304a18a79967f8a53ba" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa95fcf2448c4166902c56d01332630c"
I20260809 22:52:33.394273 18766 raft_consensus.cc:3037] T 61b0b10f318a4d169acd393b1b331de4 P fa95fcf2448c4166902c56d01332630c [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:52:33.394325 18976 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "61b0b10f318a4d169acd393b1b331de4" candidate_uuid: "8e8fcf7e414b4304a18a79967f8a53ba" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0c75510608d24ff78855de400475c524"
I20260809 22:52:33.394510 18976 raft_consensus.cc:3037] T 61b0b10f318a4d169acd393b1b331de4 P 0c75510608d24ff78855de400475c524 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:52:33.396311 18766 raft_consensus.cc:2454] T 61b0b10f318a4d169acd393b1b331de4 P fa95fcf2448c4166902c56d01332630c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8e8fcf7e414b4304a18a79967f8a53ba in term 1.
I20260809 22:52:33.396739 19093 leader_election.cc:302] T 61b0b10f318a4d169acd393b1b331de4 P 8e8fcf7e414b4304a18a79967f8a53ba [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8e8fcf7e414b4304a18a79967f8a53ba, fa95fcf2448c4166902c56d01332630c; no voters:
I20260809 22:52:33.397006 19324 raft_consensus.cc:2781] T 61b0b10f318a4d169acd393b1b331de4 P 8e8fcf7e414b4304a18a79967f8a53ba [term 1 FOLLOWER]: Leader election won for term 1
I20260809 22:52:33.397452 19324 raft_consensus.cc:686] T 61b0b10f318a4d169acd393b1b331de4 P 8e8fcf7e414b4304a18a79967f8a53ba [term 1 LEADER]: Becoming Leader. State: Replica: 8e8fcf7e414b4304a18a79967f8a53ba, State: Running, Role: LEADER
I20260809 22:52:33.397598 18976 raft_consensus.cc:2454] T 61b0b10f318a4d169acd393b1b331de4 P 0c75510608d24ff78855de400475c524 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8e8fcf7e414b4304a18a79967f8a53ba in term 1.
I20260809 22:52:33.397954 19324 consensus_queue.cc:237] T 61b0b10f318a4d169acd393b1b331de4 P 8e8fcf7e414b4304a18a79967f8a53ba [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa95fcf2448c4166902c56d01332630c" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 35739 } } peers { permanent_uuid: "0c75510608d24ff78855de400475c524" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 34115 } } peers { permanent_uuid: "8e8fcf7e414b4304a18a79967f8a53ba" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 44931 } }
I20260809 22:52:33.401285 18513 catalog_manager.cc:5495] T 61b0b10f318a4d169acd393b1b331de4 P 8e8fcf7e414b4304a18a79967f8a53ba reported cstate change: term changed from 0 to 1, leader changed from <none> to 8e8fcf7e414b4304a18a79967f8a53ba (127.15.113.131). New cstate: current_term: 1 leader_uuid: "8e8fcf7e414b4304a18a79967f8a53ba" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa95fcf2448c4166902c56d01332630c" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 35739 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0c75510608d24ff78855de400475c524" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 34115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8e8fcf7e414b4304a18a79967f8a53ba" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 44931 } health_report { overall_health: HEALTHY } } }
I20260809 22:52:33.461172 19412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:52:33.454749 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48748 (local address 127.15.113.131:44931)
0809 22:52:33.455333 (+ 584us) server_negotiation.cc:206] Beginning negotiation
0809 22:52:33.455339 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:52:33.455361 (+ 22us) server_negotiation.cc:407] Connection header received
0809 22:52:33.455456 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:52:33.455462 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:52:33.455556 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:52:33.455821 (+ 265us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 22:52:33.456238 (+ 417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:52:33.457044 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:52:33.458920 (+ 1876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:52:33.459312 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:52:33.459399 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:52:33.459928 (+ 529us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 22:52:33.460513 (+ 585us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 22:52:33.460637 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0809 22:52:33.460740 (+ 103us) server_negotiation.cc:299] Negotiation successful
0809 22:52:33.460803 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":467,"thread_start_us":397,"threads_started":1}
I20260809 22:52:33.476334 19303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:52:33.471317 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.113.131:51825 (local address 127.15.113.130:34115)
0809 22:52:33.471509 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0809 22:52:33.471515 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:52:33.471695 (+ 180us) server_negotiation.cc:407] Connection header received
0809 22:52:33.471883 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:52:33.471888 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:52:33.471955 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:52:33.472027 (+ 72us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 22:52:33.472556 (+ 529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:52:33.473383 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:52:33.474692 (+ 1309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:52:33.475098 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:52:33.475210 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:52:33.475612 (+ 402us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 22:52:33.475739 (+ 127us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 22:52:33.475815 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0809 22:52:33.475948 (+ 133us) server_negotiation.cc:299] Negotiation successful
0809 22:52:33.476024 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260809 22:52:33.476399 19416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:52:33.471078 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.130:34115 (local address 127.15.113.131:51825)
0809 22:52:33.471597 (+ 519us) negotiation.cc:107] Waiting for socket to connect
0809 22:52:33.471613 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 22:52:33.471756 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:52:33.472053 (+ 297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:52:33.472058 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:52:33.472069 (+ 11us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 22:52:33.472415 (+ 346us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:52:33.472425 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:52:33.473508 (+ 1083us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:52:33.473514 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:52:33.474554 (+ 1040us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:52:33.474566 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:52:33.475269 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:52:33.475276 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:52:33.475395 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:52:33.475445 (+ 50us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 22:52:33.475835 (+ 390us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 22:52:33.475847 (+ 12us) client_negotiation.cc:769] Sending connection context
0809 22:52:33.475912 (+ 65us) client_negotiation.cc:240] Negotiation successful
0809 22:52:33.476037 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":371,"thread_start_us":248,"threads_started":1}
I20260809 22:52:33.486060 18766 raft_consensus.cc:1261] T 61b0b10f318a4d169acd393b1b331de4 P fa95fcf2448c4166902c56d01332630c [term 1 FOLLOWER]: Refusing update from remote peer 8e8fcf7e414b4304a18a79967f8a53ba: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 22:52:33.486187 18976 raft_consensus.cc:1261] T 61b0b10f318a4d169acd393b1b331de4 P 0c75510608d24ff78855de400475c524 [term 1 FOLLOWER]: Refusing update from remote peer 8e8fcf7e414b4304a18a79967f8a53ba: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 22:52:33.486720 19324 consensus_queue.cc:1035] T 61b0b10f318a4d169acd393b1b331de4 P 8e8fcf7e414b4304a18a79967f8a53ba [LEADER]: Connected to new peer: Peer: permanent_uuid: "fa95fcf2448c4166902c56d01332630c" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 35739 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 22:52:33.487318 19410 consensus_queue.cc:1035] T 61b0b10f318a4d169acd393b1b331de4 P 8e8fcf7e414b4304a18a79967f8a53ba [LEADER]: Connected to new peer: Peer: permanent_uuid: "0c75510608d24ff78855de400475c524" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 34115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 22:52:33.514758 19412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:52:33.509070 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.113.130:34559 (local address 127.15.113.131:44931)
0809 22:52:33.509229 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0809 22:52:33.509235 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:52:33.509254 (+ 19us) server_negotiation.cc:407] Connection header received
0809 22:52:33.509314 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:52:33.509320 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:52:33.509394 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:52:33.509944 (+ 550us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 22:52:33.510084 (+ 140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:52:33.511149 (+ 1065us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:52:33.512579 (+ 1430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:52:33.513329 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:52:33.513452 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:52:33.514039 (+ 587us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 22:52:33.514168 (+ 129us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 22:52:33.514255 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0809 22:52:33.514417 (+ 162us) server_negotiation.cc:299] Negotiation successful
0809 22:52:33.514495 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260809 22:52:33.514868 19377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:52:33.508757 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.131:44931 (local address 127.15.113.130:34559)
0809 22:52:33.509089 (+ 332us) negotiation.cc:107] Waiting for socket to connect
0809 22:52:33.509110 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0809 22:52:33.509215 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:52:33.509553 (+ 338us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:52:33.509559 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:52:33.509572 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 22:52:33.509779 (+ 207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:52:33.509789 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:52:33.511282 (+ 1493us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:52:33.511286 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:52:33.512460 (+ 1174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:52:33.512473 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:52:33.513580 (+ 1107us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:52:33.513586 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:52:33.513825 (+ 239us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:52:33.513876 (+ 51us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 22:52:33.514275 (+ 399us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 22:52:33.514285 (+ 10us) client_negotiation.cc:769] Sending connection context
0809 22:52:33.514371 (+ 86us) client_negotiation.cc:240] Negotiation successful
0809 22:52:33.514521 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":189}
W20260809 22:52:33.515920 19356 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260809 22:52:35.143311 18511 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60638:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 22:52:35.143714 18511 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60638:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 22:52:35.149802 18511 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 72c0d959ee5c41298a2ea15ccc17437f: Sending DeleteTablet for 3 replicas of tablet 61b0b10f318a4d169acd393b1b331de4
I20260809 22:52:35.150805 18729 tablet_service.cc:1514] Processing DeleteTablet for tablet 61b0b10f318a4d169acd393b1b331de4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:52:35 UTC) from {username='kudu', principal='kudu/127.15.113.190@KRBTEST.COM'} at 127.0.0.1:34790
I20260809 22:52:35.150833 18945 tablet_service.cc:1514] Processing DeleteTablet for tablet 61b0b10f318a4d169acd393b1b331de4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:52:35 UTC) from {username='kudu', principal='kudu/127.15.113.190@KRBTEST.COM'} at 127.0.0.1:38374
I20260809 22:52:35.150849 19161 tablet_service.cc:1514] Processing DeleteTablet for tablet 61b0b10f318a4d169acd393b1b331de4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:52:35 UTC) from {username='kudu', principal='kudu/127.15.113.190@KRBTEST.COM'} at 127.0.0.1:48732
I20260809 22:52:35.162251 19455 tablet_replica.cc:330] T 61b0b10f318a4d169acd393b1b331de4 P fa95fcf2448c4166902c56d01332630c: stopping tablet replica
I20260809 22:52:35.163285 19455 raft_consensus.cc:2229] T 61b0b10f318a4d169acd393b1b331de4 P fa95fcf2448c4166902c56d01332630c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 22:52:35.163877 19455 raft_consensus.cc:2258] T 61b0b10f318a4d169acd393b1b331de4 P fa95fcf2448c4166902c56d01332630c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 22:52:35.165966 19454 tablet_replica.cc:330] T 61b0b10f318a4d169acd393b1b331de4 P 8e8fcf7e414b4304a18a79967f8a53ba: stopping tablet replica
I20260809 22:52:35.166118 19453 tablet_replica.cc:330] T 61b0b10f318a4d169acd393b1b331de4 P 0c75510608d24ff78855de400475c524: stopping tablet replica
I20260809 22:52:35.166246 19454 raft_consensus.cc:2229] T 61b0b10f318a4d169acd393b1b331de4 P 8e8fcf7e414b4304a18a79967f8a53ba [term 1 LEADER]: Raft consensus shutting down.
I20260809 22:52:35.166450 19453 raft_consensus.cc:2229] T 61b0b10f318a4d169acd393b1b331de4 P 0c75510608d24ff78855de400475c524 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 22:52:35.166900 19454 raft_consensus.cc:2258] T 61b0b10f318a4d169acd393b1b331de4 P 8e8fcf7e414b4304a18a79967f8a53ba [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 22:52:35.166914 19453 raft_consensus.cc:2258] T 61b0b10f318a4d169acd393b1b331de4 P 0c75510608d24ff78855de400475c524 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 22:52:35.167496 19455 ts_tablet_manager.cc:1905] T 61b0b10f318a4d169acd393b1b331de4 P fa95fcf2448c4166902c56d01332630c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 22:52:35.167896 19454 ts_tablet_manager.cc:1905] T 61b0b10f318a4d169acd393b1b331de4 P 8e8fcf7e414b4304a18a79967f8a53ba: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 22:52:35.168006 19453 ts_tablet_manager.cc:1905] T 61b0b10f318a4d169acd393b1b331de4 P 0c75510608d24ff78855de400475c524: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 22:52:35.176268 19453 ts_tablet_manager.cc:1918] T 61b0b10f318a4d169acd393b1b331de4 P 0c75510608d24ff78855de400475c524: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260809 22:52:35.176262 19454 ts_tablet_manager.cc:1918] T 61b0b10f318a4d169acd393b1b331de4 P 8e8fcf7e414b4304a18a79967f8a53ba: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260809 22:52:35.176321 19455 ts_tablet_manager.cc:1918] T 61b0b10f318a4d169acd393b1b331de4 P fa95fcf2448c4166902c56d01332630c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260809 22:52:35.176491 19453 log.cc:1196] T 61b0b10f318a4d169acd393b1b331de4 P 0c75510608d24ff78855de400475c524: Deleting WAL directory at /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315935617919-15814-0/minicluster-data/ts-1/wal/wals/61b0b10f318a4d169acd393b1b331de4
I20260809 22:52:35.176494 19454 log.cc:1196] T 61b0b10f318a4d169acd393b1b331de4 P 8e8fcf7e414b4304a18a79967f8a53ba: Deleting WAL directory at /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315935617919-15814-0/minicluster-data/ts-2/wal/wals/61b0b10f318a4d169acd393b1b331de4
I20260809 22:52:35.176716 19455 log.cc:1196] T 61b0b10f318a4d169acd393b1b331de4 P fa95fcf2448c4166902c56d01332630c: Deleting WAL directory at /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315935617919-15814-0/minicluster-data/ts-0/wal/wals/61b0b10f318a4d169acd393b1b331de4
I20260809 22:52:35.177177 19454 ts_tablet_manager.cc:1939] T 61b0b10f318a4d169acd393b1b331de4 P 8e8fcf7e414b4304a18a79967f8a53ba: Deleting consensus metadata
I20260809 22:52:35.177317 19453 ts_tablet_manager.cc:1939] T 61b0b10f318a4d169acd393b1b331de4 P 0c75510608d24ff78855de400475c524: Deleting consensus metadata
I20260809 22:52:35.177501 19455 ts_tablet_manager.cc:1939] T 61b0b10f318a4d169acd393b1b331de4 P fa95fcf2448c4166902c56d01332630c: Deleting consensus metadata
I20260809 22:52:35.178514 18487 catalog_manager.cc:4841] TS 8e8fcf7e414b4304a18a79967f8a53ba (127.15.113.131:44931): tablet 61b0b10f318a4d169acd393b1b331de4 (table test-table [id=ebeb248942b34470809e13a26df1133a]) successfully deleted
I20260809 22:52:35.178581 18491 catalog_manager.cc:4841] TS 0c75510608d24ff78855de400475c524 (127.15.113.130:34115): tablet 61b0b10f318a4d169acd393b1b331de4 (table test-table [id=ebeb248942b34470809e13a26df1133a]) successfully deleted
I20260809 22:52:35.178933 18491 catalog_manager.cc:4841] TS fa95fcf2448c4166902c56d01332630c (127.15.113.129:35739): tablet 61b0b10f318a4d169acd393b1b331de4 (table test-table [id=ebeb248942b34470809e13a26df1133a]) successfully deleted
Aug 09 22:52:35 dist-test-slave-ncp9 krb5kdc[18445](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315955, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 09 22:52:35 dist-test-slave-ncp9 krb5kdc[18445](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315955, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.15.113.190@KRBTEST.COM
I20260809 22:52:35.218246 19465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:52:35.204627 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60654 (local address 127.15.113.190:45749)
0809 22:52:35.205205 (+ 578us) server_negotiation.cc:206] Beginning negotiation
0809 22:52:35.205210 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:52:35.205268 (+ 58us) server_negotiation.cc:407] Connection header received
0809 22:52:35.205487 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:52:35.205491 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:52:35.205556 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:52:35.205656 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:52:35.206963 (+ 1307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:52:35.207703 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:52:35.208973 (+ 1270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:52:35.209368 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:52:35.209490 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:52:35.213280 (+ 3790us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:52:35.213285 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:52:35.213288 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:52:35.213319 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:52:35.215786 (+ 2467us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:52:35.216396 (+ 610us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:52:35.216400 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:52:35.216402 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:52:35.216528 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:52:35.216864 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:52:35.216868 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:52:35.216870 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:52:35.217172 (+ 302us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:52:35.217412 (+ 240us) server_negotiation.cc:1035] Waiting for connection context
0809 22:52:35.217687 (+ 275us) server_negotiation.cc:299] Negotiation successful
0809 22:52:35.217894 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":448,"thread_start_us":264,"threads_started":1}
I20260809 22:52:35.221390 18511 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60654:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 22:52:35.221931 18511 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 22:52:35.227850 18729 tablet_service.cc:1467] Processing CreateTablet for tablet 6d9b52b76cff4b4ab67a7c11c5fd6cb7 (DEFAULT_TABLE table=test-table [id=ecbcaca9f73f493f82e86b4419091fd3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:52:35.228113 18945 tablet_service.cc:1467] Processing CreateTablet for tablet 6d9b52b76cff4b4ab67a7c11c5fd6cb7 (DEFAULT_TABLE table=test-table [id=ecbcaca9f73f493f82e86b4419091fd3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:52:35.228173 19161 tablet_service.cc:1467] Processing CreateTablet for tablet 6d9b52b76cff4b4ab67a7c11c5fd6cb7 (DEFAULT_TABLE table=test-table [id=ecbcaca9f73f493f82e86b4419091fd3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:52:35.228333 18729 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6d9b52b76cff4b4ab67a7c11c5fd6cb7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:52:35.228508 18945 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6d9b52b76cff4b4ab67a7c11c5fd6cb7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:52:35.228565 19161 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6d9b52b76cff4b4ab67a7c11c5fd6cb7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:52:35.238435 19466 tablet_bootstrap.cc:492] T 6d9b52b76cff4b4ab67a7c11c5fd6cb7 P 0c75510608d24ff78855de400475c524: Bootstrap starting.
I20260809 22:52:35.239594 19469 tablet_bootstrap.cc:492] T 6d9b52b76cff4b4ab67a7c11c5fd6cb7 P 8e8fcf7e414b4304a18a79967f8a53ba: Bootstrap starting.
I20260809 22:52:35.241845 19466 tablet_bootstrap.cc:654] T 6d9b52b76cff4b4ab67a7c11c5fd6cb7 P 0c75510608d24ff78855de400475c524: Neither blocks nor log segments found. Creating new log.
I20260809 22:52:35.243011 19469 tablet_bootstrap.cc:654] T 6d9b52b76cff4b4ab67a7c11c5fd6cb7 P 8e8fcf7e414b4304a18a79967f8a53ba: Neither blocks nor log segments found. Creating new log.
I20260809 22:52:35.245256 19467 tablet_bootstrap.cc:492] T 6d9b52b76cff4b4ab67a7c11c5fd6cb7 P fa95fcf2448c4166902c56d01332630c: Bootstrap starting.
I20260809 22:52:35.246071 19466 tablet_bootstrap.cc:492] T 6d9b52b76cff4b4ab67a7c11c5fd6cb7 P 0c75510608d24ff78855de400475c524: No bootstrap required, opened a new log
I20260809 22:52:35.246379 19466 ts_tablet_manager.cc:1397] T 6d9b52b76cff4b4ab67a7c11c5fd6cb7 P 0c75510608d24ff78855de400475c524: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.007s
I20260809 22:52:35.247478 19469 tablet_bootstrap.cc:492] T 6d9b52b76cff4b4ab67a7c11c5fd6cb7 P 8e8fcf7e414b4304a18a79967f8a53ba: No bootstrap required, opened a new log
I20260809 22:52:35.247716 19469 ts_tablet_manager.cc:1397] T 6d9b52b76cff4b4ab67a7c11c5fd6cb7 P 8e8fcf7e414b4304a18a79967f8a53ba: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.007s
I20260809 22:52:35.247605 19466 raft_consensus.cc:348] T 6d9b52b76cff4b4ab67a7c11c5fd6cb7 P 0c75510608d24ff78855de400475c524 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa95fcf2448c4166902c56d01332630c" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 35739 } } peers { permanent_uuid: "0c75510608d24ff78855de400475c524" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 34115 } } peers { permanent_uuid: "8e8fcf7e414b4304a18a79967f8a53ba" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 44931 } }
I20260809 22:52:35.248108 19466 raft_consensus.cc:374] T 6d9b52b76cff4b4ab67a7c11c5fd6cb7 P 0c75510608d24ff78855de400475c524 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:52:35.248255 19467 tablet_bootstrap.cc:654] T 6d9b52b76cff4b4ab67a7c11c5fd6cb7 P fa95fcf2448c4166902c56d01332630c: Neither blocks nor log segments found. Creating new log.
I20260809 22:52:35.248353 19466 raft_consensus.cc:729] T 6d9b52b76cff4b4ab67a7c11c5fd6cb7 P 0c75510608d24ff78855de400475c524 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0c75510608d24ff78855de400475c524, State: Initialized, Role: FOLLOWER
I20260809 22:52:35.248725 19466 consensus_queue.cc:260] T 6d9b52b76cff4b4ab67a7c11c5fd6cb7 P 0c75510608d24ff78855de400475c524 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa95fcf2448c4166902c56d01332630c" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 35739 } } peers { permanent_uuid: "0c75510608d24ff78855de400475c524" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 34115 } } peers { permanent_uuid: "8e8fcf7e414b4304a18a79967f8a53ba" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 44931 } }
I20260809 22:52:35.249229 19469 raft_consensus.cc:348] T 6d9b52b76cff4b4ab67a7c11c5fd6cb7 P 8e8fcf7e414b4304a18a79967f8a53ba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa95fcf2448c4166902c56d01332630c" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 35739 } } peers { permanent_uuid: "0c75510608d24ff78855de400475c524" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 34115 } } peers { permanent_uuid: "8e8fcf7e414b4304a18a79967f8a53ba" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 44931 } }
I20260809 22:52:35.249639 19466 ts_tablet_manager.cc:1428] T 6d9b52b76cff4b4ab67a7c11c5fd6cb7 P 0c75510608d24ff78855de400475c524: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
I20260809 22:52:35.249728 19469 raft_consensus.cc:374] T 6d9b52b76cff4b4ab67a7c11c5fd6cb7 P 8e8fcf7e414b4304a18a79967f8a53ba [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:52:35.250027 19469 raft_consensus.cc:729] T 6d9b52b76cff4b4ab67a7c11c5fd6cb7 P 8e8fcf7e414b4304a18a79967f8a53ba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8e8fcf7e414b4304a18a79967f8a53ba, State: Initialized, Role: FOLLOWER
I20260809 22:52:35.250452 19469 consensus_queue.cc:260] T 6d9b52b76cff4b4ab67a7c11c5fd6cb7 P 8e8fcf7e414b4304a18a79967f8a53ba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa95fcf2448c4166902c56d01332630c" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 35739 } } peers { permanent_uuid: "0c75510608d24ff78855de400475c524" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 34115 } } peers { permanent_uuid: "8e8fcf7e414b4304a18a79967f8a53ba" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 44931 } }
I20260809 22:52:35.251298 19469 ts_tablet_manager.cc:1428] T 6d9b52b76cff4b4ab67a7c11c5fd6cb7 P 8e8fcf7e414b4304a18a79967f8a53ba: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 22:52:35.252116 19467 tablet_bootstrap.cc:492] T 6d9b52b76cff4b4ab67a7c11c5fd6cb7 P fa95fcf2448c4166902c56d01332630c: No bootstrap required, opened a new log
I20260809 22:52:35.252357 19467 ts_tablet_manager.cc:1397] T 6d9b52b76cff4b4ab67a7c11c5fd6cb7 P fa95fcf2448c4166902c56d01332630c: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.006s
I20260809 22:52:35.254456 19467 raft_consensus.cc:348] T 6d9b52b76cff4b4ab67a7c11c5fd6cb7 P fa95fcf2448c4166902c56d01332630c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa95fcf2448c4166902c56d01332630c" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 35739 } } peers { permanent_uuid: "0c75510608d24ff78855de400475c524" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 34115 } } peers { permanent_uuid: "8e8fcf7e414b4304a18a79967f8a53ba" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 44931 } }
I20260809 22:52:35.254665 19467 raft_consensus.cc:374] T 6d9b52b76cff4b4ab67a7c11c5fd6cb7 P fa95fcf2448c4166902c56d01332630c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:52:35.254873 19467 raft_consensus.cc:729] T 6d9b52b76cff4b4ab67a7c11c5fd6cb7 P fa95fcf2448c4166902c56d01332630c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa95fcf2448c4166902c56d01332630c, State: Initialized, Role: FOLLOWER
I20260809 22:52:35.255225 19467 consensus_queue.cc:260] T 6d9b52b76cff4b4ab67a7c11c5fd6cb7 P fa95fcf2448c4166902c56d01332630c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa95fcf2448c4166902c56d01332630c" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 35739 } } peers { permanent_uuid: "0c75510608d24ff78855de400475c524" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 34115 } } peers { permanent_uuid: "8e8fcf7e414b4304a18a79967f8a53ba" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 44931 } }
I20260809 22:52:35.256269 19467 ts_tablet_manager.cc:1428] T 6d9b52b76cff4b4ab67a7c11c5fd6cb7 P fa95fcf2448c4166902c56d01332630c: Time spent starting tablet: real 0.004s user 0.000s sys 0.003s
I20260809 22:52:35.293965 19444 raft_consensus.cc:482] T 6d9b52b76cff4b4ab67a7c11c5fd6cb7 P 8e8fcf7e414b4304a18a79967f8a53ba [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 22:52:35.294229 19444 raft_consensus.cc:504] T 6d9b52b76cff4b4ab67a7c11c5fd6cb7 P 8e8fcf7e414b4304a18a79967f8a53ba [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa95fcf2448c4166902c56d01332630c" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 35739 } } peers { permanent_uuid: "0c75510608d24ff78855de400475c524" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 34115 } } peers { permanent_uuid: "8e8fcf7e414b4304a18a79967f8a53ba" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 44931 } }
I20260809 22:52:35.294973 19444 leader_election.cc:288] T 6d9b52b76cff4b4ab67a7c11c5fd6cb7 P 8e8fcf7e414b4304a18a79967f8a53ba [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fa95fcf2448c4166902c56d01332630c (127.15.113.129:35739), 0c75510608d24ff78855de400475c524 (127.15.113.130:34115)
I20260809 22:52:35.295673 18767 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6d9b52b76cff4b4ab67a7c11c5fd6cb7" candidate_uuid: "8e8fcf7e414b4304a18a79967f8a53ba" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa95fcf2448c4166902c56d01332630c" is_pre_election: true
I20260809 22:52:35.295720 18976 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6d9b52b76cff4b4ab67a7c11c5fd6cb7" candidate_uuid: "8e8fcf7e414b4304a18a79967f8a53ba" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0c75510608d24ff78855de400475c524" is_pre_election: true
I20260809 22:52:35.295985 18767 raft_consensus.cc:2454] T 6d9b52b76cff4b4ab67a7c11c5fd6cb7 P fa95fcf2448c4166902c56d01332630c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e8fcf7e414b4304a18a79967f8a53ba in term 0.
I20260809 22:52:35.296105 18976 raft_consensus.cc:2454] T 6d9b52b76cff4b4ab67a7c11c5fd6cb7 P 0c75510608d24ff78855de400475c524 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e8fcf7e414b4304a18a79967f8a53ba in term 0.
I20260809 22:52:35.296589 19093 leader_election.cc:302] T 6d9b52b76cff4b4ab67a7c11c5fd6cb7 P 8e8fcf7e414b4304a18a79967f8a53ba [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8e8fcf7e414b4304a18a79967f8a53ba, fa95fcf2448c4166902c56d01332630c; no voters:
I20260809 22:52:35.296958 19444 raft_consensus.cc:2781] T 6d9b52b76cff4b4ab67a7c11c5fd6cb7 P 8e8fcf7e414b4304a18a79967f8a53ba [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 22:52:35.297152 19444 raft_consensus.cc:482] T 6d9b52b76cff4b4ab67a7c11c5fd6cb7 P 8e8fcf7e414b4304a18a79967f8a53ba [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 22:52:35.297281 19444 raft_consensus.cc:3037] T 6d9b52b76cff4b4ab67a7c11c5fd6cb7 P 8e8fcf7e414b4304a18a79967f8a53ba [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:52:35.299960 19444 raft_consensus.cc:504] T 6d9b52b76cff4b4ab67a7c11c5fd6cb7 P 8e8fcf7e414b4304a18a79967f8a53ba [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa95fcf2448c4166902c56d01332630c" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 35739 } } peers { permanent_uuid: "0c75510608d24ff78855de400475c524" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 34115 } } peers { permanent_uuid: "8e8fcf7e414b4304a18a79967f8a53ba" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 44931 } }
I20260809 22:52:35.300575 19444 leader_election.cc:288] T 6d9b52b76cff4b4ab67a7c11c5fd6cb7 P 8e8fcf7e414b4304a18a79967f8a53ba [CANDIDATE]: Term 1 election: Requested vote from peers fa95fcf2448c4166902c56d01332630c (127.15.113.129:35739), 0c75510608d24ff78855de400475c524 (127.15.113.130:34115)
I20260809 22:52:35.301059 18767 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6d9b52b76cff4b4ab67a7c11c5fd6cb7" candidate_uuid: "8e8fcf7e414b4304a18a79967f8a53ba" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa95fcf2448c4166902c56d01332630c"
I20260809 22:52:35.301211 18976 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6d9b52b76cff4b4ab67a7c11c5fd6cb7" candidate_uuid: "8e8fcf7e414b4304a18a79967f8a53ba" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0c75510608d24ff78855de400475c524"
I20260809 22:52:35.301434 18976 raft_consensus.cc:3037] T 6d9b52b76cff4b4ab67a7c11c5fd6cb7 P 0c75510608d24ff78855de400475c524 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:52:35.301457 18767 raft_consensus.cc:3037] T 6d9b52b76cff4b4ab67a7c11c5fd6cb7 P fa95fcf2448c4166902c56d01332630c [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:52:35.304380 18976 raft_consensus.cc:2454] T 6d9b52b76cff4b4ab67a7c11c5fd6cb7 P 0c75510608d24ff78855de400475c524 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8e8fcf7e414b4304a18a79967f8a53ba in term 1.
I20260809 22:52:35.304479 18767 raft_consensus.cc:2454] T 6d9b52b76cff4b4ab67a7c11c5fd6cb7 P fa95fcf2448c4166902c56d01332630c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8e8fcf7e414b4304a18a79967f8a53ba in term 1.
I20260809 22:52:35.304821 19093 leader_election.cc:302] T 6d9b52b76cff4b4ab67a7c11c5fd6cb7 P 8e8fcf7e414b4304a18a79967f8a53ba [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0c75510608d24ff78855de400475c524, 8e8fcf7e414b4304a18a79967f8a53ba; no voters:
I20260809 22:52:35.305259 19444 raft_consensus.cc:2781] T 6d9b52b76cff4b4ab67a7c11c5fd6cb7 P 8e8fcf7e414b4304a18a79967f8a53ba [term 1 FOLLOWER]: Leader election won for term 1
I20260809 22:52:35.305414 19444 raft_consensus.cc:686] T 6d9b52b76cff4b4ab67a7c11c5fd6cb7 P 8e8fcf7e414b4304a18a79967f8a53ba [term 1 LEADER]: Becoming Leader. State: Replica: 8e8fcf7e414b4304a18a79967f8a53ba, State: Running, Role: LEADER
I20260809 22:52:35.305752 19444 consensus_queue.cc:237] T 6d9b52b76cff4b4ab67a7c11c5fd6cb7 P 8e8fcf7e414b4304a18a79967f8a53ba [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa95fcf2448c4166902c56d01332630c" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 35739 } } peers { permanent_uuid: "0c75510608d24ff78855de400475c524" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 34115 } } peers { permanent_uuid: "8e8fcf7e414b4304a18a79967f8a53ba" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 44931 } }
I20260809 22:52:35.308720 18514 catalog_manager.cc:5495] T 6d9b52b76cff4b4ab67a7c11c5fd6cb7 P 8e8fcf7e414b4304a18a79967f8a53ba reported cstate change: term changed from 0 to 1, leader changed from <none> to 8e8fcf7e414b4304a18a79967f8a53ba (127.15.113.131). New cstate: current_term: 1 leader_uuid: "8e8fcf7e414b4304a18a79967f8a53ba" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa95fcf2448c4166902c56d01332630c" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 35739 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0c75510608d24ff78855de400475c524" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 34115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8e8fcf7e414b4304a18a79967f8a53ba" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 44931 } health_report { overall_health: HEALTHY } } }
I20260809 22:52:35.345901 19480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:52:35.338825 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48752 (local address 127.15.113.131:44931)
0809 22:52:35.340039 (+ 1214us) server_negotiation.cc:206] Beginning negotiation
0809 22:52:35.340044 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:52:35.340081 (+ 37us) server_negotiation.cc:407] Connection header received
0809 22:52:35.340130 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:52:35.340133 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:52:35.340189 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:52:35.340267 (+ 78us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 22:52:35.340776 (+ 509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:52:35.342070 (+ 1294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:52:35.343520 (+ 1450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:52:35.344220 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:52:35.344312 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:52:35.345035 (+ 723us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 22:52:35.345206 (+ 171us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 22:52:35.345273 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0809 22:52:35.345381 (+ 108us) server_negotiation.cc:299] Negotiation successful
0809 22:52:35.345441 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1047,"thread_start_us":810,"threads_started":1}
I20260809 22:52:35.354827 18976 raft_consensus.cc:1261] T 6d9b52b76cff4b4ab67a7c11c5fd6cb7 P 0c75510608d24ff78855de400475c524 [term 1 FOLLOWER]: Refusing update from remote peer 8e8fcf7e414b4304a18a79967f8a53ba: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 22:52:35.354839 18767 raft_consensus.cc:1261] T 6d9b52b76cff4b4ab67a7c11c5fd6cb7 P fa95fcf2448c4166902c56d01332630c [term 1 FOLLOWER]: Refusing update from remote peer 8e8fcf7e414b4304a18a79967f8a53ba: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 22:52:35.355448 19324 consensus_queue.cc:1035] T 6d9b52b76cff4b4ab67a7c11c5fd6cb7 P 8e8fcf7e414b4304a18a79967f8a53ba [LEADER]: Connected to new peer: Peer: permanent_uuid: "0c75510608d24ff78855de400475c524" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 34115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 22:52:35.355770 19444 consensus_queue.cc:1035] T 6d9b52b76cff4b4ab67a7c11c5fd6cb7 P 8e8fcf7e414b4304a18a79967f8a53ba [LEADER]: Connected to new peer: Peer: permanent_uuid: "fa95fcf2448c4166902c56d01332630c" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 35739 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 22:52:35.401661 18514 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60654:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 22:52:35.402051 18514 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60654:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 22:52:35.405186 18514 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 72c0d959ee5c41298a2ea15ccc17437f: Sending DeleteTablet for 3 replicas of tablet 6d9b52b76cff4b4ab67a7c11c5fd6cb7
I20260809 22:52:35.406167 18729 tablet_service.cc:1514] Processing DeleteTablet for tablet 6d9b52b76cff4b4ab67a7c11c5fd6cb7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:52:35 UTC) from {username='kudu', principal='kudu/127.15.113.190@KRBTEST.COM'} at 127.0.0.1:34790
I20260809 22:52:35.406201 19161 tablet_service.cc:1514] Processing DeleteTablet for tablet 6d9b52b76cff4b4ab67a7c11c5fd6cb7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:52:35 UTC) from {username='kudu', principal='kudu/127.15.113.190@KRBTEST.COM'} at 127.0.0.1:48732
I20260809 22:52:35.406270 18945 tablet_service.cc:1514] Processing DeleteTablet for tablet 6d9b52b76cff4b4ab67a7c11c5fd6cb7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:52:35 UTC) from {username='kudu', principal='kudu/127.15.113.190@KRBTEST.COM'} at 127.0.0.1:38374
I20260809 22:52:35.406493 19455 tablet_replica.cc:330] T 6d9b52b76cff4b4ab67a7c11c5fd6cb7 P fa95fcf2448c4166902c56d01332630c: stopping tablet replica
I20260809 22:52:35.406533 19454 tablet_replica.cc:330] T 6d9b52b76cff4b4ab67a7c11c5fd6cb7 P 8e8fcf7e414b4304a18a79967f8a53ba: stopping tablet replica
I20260809 22:52:35.406735 19453 tablet_replica.cc:330] T 6d9b52b76cff4b4ab67a7c11c5fd6cb7 P 0c75510608d24ff78855de400475c524: stopping tablet replica
I20260809 22:52:35.406764 19455 raft_consensus.cc:2229] T 6d9b52b76cff4b4ab67a7c11c5fd6cb7 P fa95fcf2448c4166902c56d01332630c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 22:52:35.406800 19454 raft_consensus.cc:2229] T 6d9b52b76cff4b4ab67a7c11c5fd6cb7 P 8e8fcf7e414b4304a18a79967f8a53ba [term 1 LEADER]: Raft consensus shutting down.
I20260809 22:52:35.406948 19455 raft_consensus.cc:2258] T 6d9b52b76cff4b4ab67a7c11c5fd6cb7 P fa95fcf2448c4166902c56d01332630c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 22:52:35.406982 19453 raft_consensus.cc:2229] T 6d9b52b76cff4b4ab67a7c11c5fd6cb7 P 0c75510608d24ff78855de400475c524 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 22:52:35.407079 19454 raft_consensus.cc:2258] T 6d9b52b76cff4b4ab67a7c11c5fd6cb7 P 8e8fcf7e414b4304a18a79967f8a53ba [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 22:52:35.407177 19453 raft_consensus.cc:2258] T 6d9b52b76cff4b4ab67a7c11c5fd6cb7 P 0c75510608d24ff78855de400475c524 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 22:52:35.407541 18514 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60654:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 22:52:35.408179 19454 ts_tablet_manager.cc:1905] T 6d9b52b76cff4b4ab67a7c11c5fd6cb7 P 8e8fcf7e414b4304a18a79967f8a53ba: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260809 22:52:35.408298 18514 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 22:52:35.408432 19455 ts_tablet_manager.cc:1905] T 6d9b52b76cff4b4ab67a7c11c5fd6cb7 P fa95fcf2448c4166902c56d01332630c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 22:52:35.408553 19453 ts_tablet_manager.cc:1905] T 6d9b52b76cff4b4ab67a7c11c5fd6cb7 P 0c75510608d24ff78855de400475c524: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 22:52:35.415616 19453 ts_tablet_manager.cc:1918] T 6d9b52b76cff4b4ab67a7c11c5fd6cb7 P 0c75510608d24ff78855de400475c524: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 22:52:35.415865 19453 log.cc:1196] T 6d9b52b76cff4b4ab67a7c11c5fd6cb7 P 0c75510608d24ff78855de400475c524: Deleting WAL directory at /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315935617919-15814-0/minicluster-data/ts-1/wal/wals/6d9b52b76cff4b4ab67a7c11c5fd6cb7
I20260809 22:52:35.416179 19161 tablet_service.cc:1467] Processing CreateTablet for tablet 45e33723a3724e9f819ddd48688277f1 (DEFAULT_TABLE table=test-table [id=7252eb14b5be4af3bdda8e98c253c1e4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:52:35.416374 19453 ts_tablet_manager.cc:1939] T 6d9b52b76cff4b4ab67a7c11c5fd6cb7 P 0c75510608d24ff78855de400475c524: Deleting consensus metadata
I20260809 22:52:35.416522 18729 tablet_service.cc:1467] Processing CreateTablet for tablet 45e33723a3724e9f819ddd48688277f1 (DEFAULT_TABLE table=test-table [id=7252eb14b5be4af3bdda8e98c253c1e4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:52:35.416567 18945 tablet_service.cc:1467] Processing CreateTablet for tablet 45e33723a3724e9f819ddd48688277f1 (DEFAULT_TABLE table=test-table [id=7252eb14b5be4af3bdda8e98c253c1e4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:52:35.416720 19161 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 45e33723a3724e9f819ddd48688277f1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:52:35.416920 19455 ts_tablet_manager.cc:1918] T 6d9b52b76cff4b4ab67a7c11c5fd6cb7 P fa95fcf2448c4166902c56d01332630c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 22:52:35.416976 18945 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 45e33723a3724e9f819ddd48688277f1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:52:35.417096 18729 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 45e33723a3724e9f819ddd48688277f1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:52:35.417830 19455 log.cc:1196] T 6d9b52b76cff4b4ab67a7c11c5fd6cb7 P fa95fcf2448c4166902c56d01332630c: Deleting WAL directory at /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315935617919-15814-0/minicluster-data/ts-0/wal/wals/6d9b52b76cff4b4ab67a7c11c5fd6cb7
I20260809 22:52:35.418310 19455 ts_tablet_manager.cc:1939] T 6d9b52b76cff4b4ab67a7c11c5fd6cb7 P fa95fcf2448c4166902c56d01332630c: Deleting consensus metadata
I20260809 22:52:35.419427 18491 catalog_manager.cc:4841] TS fa95fcf2448c4166902c56d01332630c (127.15.113.129:35739): tablet 6d9b52b76cff4b4ab67a7c11c5fd6cb7 (table test-table [id=ecbcaca9f73f493f82e86b4419091fd3]) successfully deleted
I20260809 22:52:35.421842 18491 catalog_manager.cc:4841] TS 0c75510608d24ff78855de400475c524 (127.15.113.130:34115): tablet 6d9b52b76cff4b4ab67a7c11c5fd6cb7 (table test-table [id=ecbcaca9f73f493f82e86b4419091fd3]) successfully deleted
I20260809 22:52:35.423943 19454 ts_tablet_manager.cc:1918] T 6d9b52b76cff4b4ab67a7c11c5fd6cb7 P 8e8fcf7e414b4304a18a79967f8a53ba: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 22:52:35.424258 19454 log.cc:1196] T 6d9b52b76cff4b4ab67a7c11c5fd6cb7 P 8e8fcf7e414b4304a18a79967f8a53ba: Deleting WAL directory at /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315935617919-15814-0/minicluster-data/ts-2/wal/wals/6d9b52b76cff4b4ab67a7c11c5fd6cb7
I20260809 22:52:35.424266 19466 tablet_bootstrap.cc:492] T 45e33723a3724e9f819ddd48688277f1 P 0c75510608d24ff78855de400475c524: Bootstrap starting.
I20260809 22:52:35.424358 19467 tablet_bootstrap.cc:492] T 45e33723a3724e9f819ddd48688277f1 P fa95fcf2448c4166902c56d01332630c: Bootstrap starting.
I20260809 22:52:35.425289 19454 ts_tablet_manager.cc:1939] T 6d9b52b76cff4b4ab67a7c11c5fd6cb7 P 8e8fcf7e414b4304a18a79967f8a53ba: Deleting consensus metadata
I20260809 22:52:35.425979 19467 tablet_bootstrap.cc:654] T 45e33723a3724e9f819ddd48688277f1 P fa95fcf2448c4166902c56d01332630c: Neither blocks nor log segments found. Creating new log.
I20260809 22:52:35.425997 19466 tablet_bootstrap.cc:654] T 45e33723a3724e9f819ddd48688277f1 P 0c75510608d24ff78855de400475c524: Neither blocks nor log segments found. Creating new log.
I20260809 22:52:35.426537 18487 catalog_manager.cc:4841] TS 8e8fcf7e414b4304a18a79967f8a53ba (127.15.113.131:44931): tablet 6d9b52b76cff4b4ab67a7c11c5fd6cb7 (table test-table [id=ecbcaca9f73f493f82e86b4419091fd3]) successfully deleted
I20260809 22:52:35.426635 19469 tablet_bootstrap.cc:492] T 45e33723a3724e9f819ddd48688277f1 P 8e8fcf7e414b4304a18a79967f8a53ba: Bootstrap starting.
I20260809 22:52:35.428840 19469 tablet_bootstrap.cc:654] T 45e33723a3724e9f819ddd48688277f1 P 8e8fcf7e414b4304a18a79967f8a53ba: Neither blocks nor log segments found. Creating new log.
I20260809 22:52:35.435084 19467 tablet_bootstrap.cc:492] T 45e33723a3724e9f819ddd48688277f1 P fa95fcf2448c4166902c56d01332630c: No bootstrap required, opened a new log
I20260809 22:52:35.435302 19467 ts_tablet_manager.cc:1397] T 45e33723a3724e9f819ddd48688277f1 P fa95fcf2448c4166902c56d01332630c: Time spent bootstrapping tablet: real 0.011s user 0.004s sys 0.006s
I20260809 22:52:35.435489 19469 tablet_bootstrap.cc:492] T 45e33723a3724e9f819ddd48688277f1 P 8e8fcf7e414b4304a18a79967f8a53ba: No bootstrap required, opened a new log
I20260809 22:52:35.435809 19469 ts_tablet_manager.cc:1397] T 45e33723a3724e9f819ddd48688277f1 P 8e8fcf7e414b4304a18a79967f8a53ba: Time spent bootstrapping tablet: real 0.009s user 0.000s sys 0.008s
I20260809 22:52:35.436590 19467 raft_consensus.cc:348] T 45e33723a3724e9f819ddd48688277f1 P fa95fcf2448c4166902c56d01332630c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa95fcf2448c4166902c56d01332630c" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 35739 } } peers { permanent_uuid: "0c75510608d24ff78855de400475c524" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 34115 } } peers { permanent_uuid: "8e8fcf7e414b4304a18a79967f8a53ba" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 44931 } }
I20260809 22:52:35.437064 19467 raft_consensus.cc:374] T 45e33723a3724e9f819ddd48688277f1 P fa95fcf2448c4166902c56d01332630c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:52:35.437237 19467 raft_consensus.cc:729] T 45e33723a3724e9f819ddd48688277f1 P fa95fcf2448c4166902c56d01332630c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa95fcf2448c4166902c56d01332630c, State: Initialized, Role: FOLLOWER
I20260809 22:52:35.437175 19469 raft_consensus.cc:348] T 45e33723a3724e9f819ddd48688277f1 P 8e8fcf7e414b4304a18a79967f8a53ba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa95fcf2448c4166902c56d01332630c" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 35739 } } peers { permanent_uuid: "0c75510608d24ff78855de400475c524" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 34115 } } peers { permanent_uuid: "8e8fcf7e414b4304a18a79967f8a53ba" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 44931 } }
I20260809 22:52:35.437481 19466 tablet_bootstrap.cc:492] T 45e33723a3724e9f819ddd48688277f1 P 0c75510608d24ff78855de400475c524: No bootstrap required, opened a new log
I20260809 22:52:35.437706 19469 raft_consensus.cc:374] T 45e33723a3724e9f819ddd48688277f1 P 8e8fcf7e414b4304a18a79967f8a53ba [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:52:35.437765 19466 ts_tablet_manager.cc:1397] T 45e33723a3724e9f819ddd48688277f1 P 0c75510608d24ff78855de400475c524: Time spent bootstrapping tablet: real 0.013s user 0.000s sys 0.008s
I20260809 22:52:35.437719 19467 consensus_queue.cc:260] T 45e33723a3724e9f819ddd48688277f1 P fa95fcf2448c4166902c56d01332630c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa95fcf2448c4166902c56d01332630c" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 35739 } } peers { permanent_uuid: "0c75510608d24ff78855de400475c524" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 34115 } } peers { permanent_uuid: "8e8fcf7e414b4304a18a79967f8a53ba" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 44931 } }
I20260809 22:52:35.437945 19469 raft_consensus.cc:729] T 45e33723a3724e9f819ddd48688277f1 P 8e8fcf7e414b4304a18a79967f8a53ba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8e8fcf7e414b4304a18a79967f8a53ba, State: Initialized, Role: FOLLOWER
I20260809 22:52:35.438401 19469 consensus_queue.cc:260] T 45e33723a3724e9f819ddd48688277f1 P 8e8fcf7e414b4304a18a79967f8a53ba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa95fcf2448c4166902c56d01332630c" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 35739 } } peers { permanent_uuid: "0c75510608d24ff78855de400475c524" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 34115 } } peers { permanent_uuid: "8e8fcf7e414b4304a18a79967f8a53ba" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 44931 } }
I20260809 22:52:35.438570 19467 ts_tablet_manager.cc:1428] T 45e33723a3724e9f819ddd48688277f1 P fa95fcf2448c4166902c56d01332630c: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
I20260809 22:52:35.439009 19466 raft_consensus.cc:348] T 45e33723a3724e9f819ddd48688277f1 P 0c75510608d24ff78855de400475c524 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa95fcf2448c4166902c56d01332630c" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 35739 } } peers { permanent_uuid: "0c75510608d24ff78855de400475c524" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 34115 } } peers { permanent_uuid: "8e8fcf7e414b4304a18a79967f8a53ba" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 44931 } }
I20260809 22:52:35.439512 19466 raft_consensus.cc:374] T 45e33723a3724e9f819ddd48688277f1 P 0c75510608d24ff78855de400475c524 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:52:35.439684 19466 raft_consensus.cc:729] T 45e33723a3724e9f819ddd48688277f1 P 0c75510608d24ff78855de400475c524 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0c75510608d24ff78855de400475c524, State: Initialized, Role: FOLLOWER
I20260809 22:52:35.439743 19469 ts_tablet_manager.cc:1428] T 45e33723a3724e9f819ddd48688277f1 P 8e8fcf7e414b4304a18a79967f8a53ba: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260809 22:52:35.440125 19466 consensus_queue.cc:260] T 45e33723a3724e9f819ddd48688277f1 P 0c75510608d24ff78855de400475c524 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa95fcf2448c4166902c56d01332630c" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 35739 } } peers { permanent_uuid: "0c75510608d24ff78855de400475c524" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 34115 } } peers { permanent_uuid: "8e8fcf7e414b4304a18a79967f8a53ba" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 44931 } }
I20260809 22:52:35.440725 19466 ts_tablet_manager.cc:1428] T 45e33723a3724e9f819ddd48688277f1 P 0c75510608d24ff78855de400475c524: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
I20260809 22:52:35.566161 19446 raft_consensus.cc:482] T 45e33723a3724e9f819ddd48688277f1 P 0c75510608d24ff78855de400475c524 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 22:52:35.566556 19446 raft_consensus.cc:504] T 45e33723a3724e9f819ddd48688277f1 P 0c75510608d24ff78855de400475c524 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa95fcf2448c4166902c56d01332630c" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 35739 } } peers { permanent_uuid: "0c75510608d24ff78855de400475c524" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 34115 } } peers { permanent_uuid: "8e8fcf7e414b4304a18a79967f8a53ba" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 44931 } }
I20260809 22:52:35.567425 19446 leader_election.cc:288] T 45e33723a3724e9f819ddd48688277f1 P 0c75510608d24ff78855de400475c524 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fa95fcf2448c4166902c56d01332630c (127.15.113.129:35739), 8e8fcf7e414b4304a18a79967f8a53ba (127.15.113.131:44931)
I20260809 22:52:35.568147 18766 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "45e33723a3724e9f819ddd48688277f1" candidate_uuid: "0c75510608d24ff78855de400475c524" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa95fcf2448c4166902c56d01332630c" is_pre_election: true
I20260809 22:52:35.568137 19203 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "45e33723a3724e9f819ddd48688277f1" candidate_uuid: "0c75510608d24ff78855de400475c524" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e8fcf7e414b4304a18a79967f8a53ba" is_pre_election: true
I20260809 22:52:35.568524 18766 raft_consensus.cc:2454] T 45e33723a3724e9f819ddd48688277f1 P fa95fcf2448c4166902c56d01332630c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0c75510608d24ff78855de400475c524 in term 0.
I20260809 22:52:35.568529 19203 raft_consensus.cc:2454] T 45e33723a3724e9f819ddd48688277f1 P 8e8fcf7e414b4304a18a79967f8a53ba [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0c75510608d24ff78855de400475c524 in term 0.
I20260809 22:52:35.569082 18867 leader_election.cc:302] T 45e33723a3724e9f819ddd48688277f1 P 0c75510608d24ff78855de400475c524 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0c75510608d24ff78855de400475c524, fa95fcf2448c4166902c56d01332630c; no voters:
I20260809 22:52:35.569463 19446 raft_consensus.cc:2781] T 45e33723a3724e9f819ddd48688277f1 P 0c75510608d24ff78855de400475c524 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 22:52:35.569679 19446 raft_consensus.cc:482] T 45e33723a3724e9f819ddd48688277f1 P 0c75510608d24ff78855de400475c524 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 22:52:35.569762 19446 raft_consensus.cc:3037] T 45e33723a3724e9f819ddd48688277f1 P 0c75510608d24ff78855de400475c524 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:52:35.572764 19446 raft_consensus.cc:504] T 45e33723a3724e9f819ddd48688277f1 P 0c75510608d24ff78855de400475c524 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa95fcf2448c4166902c56d01332630c" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 35739 } } peers { permanent_uuid: "0c75510608d24ff78855de400475c524" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 34115 } } peers { permanent_uuid: "8e8fcf7e414b4304a18a79967f8a53ba" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 44931 } }
I20260809 22:52:35.573325 19446 leader_election.cc:288] T 45e33723a3724e9f819ddd48688277f1 P 0c75510608d24ff78855de400475c524 [CANDIDATE]: Term 1 election: Requested vote from peers fa95fcf2448c4166902c56d01332630c (127.15.113.129:35739), 8e8fcf7e414b4304a18a79967f8a53ba (127.15.113.131:44931)
I20260809 22:52:35.573944 19203 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "45e33723a3724e9f819ddd48688277f1" candidate_uuid: "0c75510608d24ff78855de400475c524" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e8fcf7e414b4304a18a79967f8a53ba"
I20260809 22:52:35.573968 18766 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "45e33723a3724e9f819ddd48688277f1" candidate_uuid: "0c75510608d24ff78855de400475c524" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa95fcf2448c4166902c56d01332630c"
I20260809 22:52:35.574235 19203 raft_consensus.cc:3037] T 45e33723a3724e9f819ddd48688277f1 P 8e8fcf7e414b4304a18a79967f8a53ba [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:52:35.574251 18766 raft_consensus.cc:3037] T 45e33723a3724e9f819ddd48688277f1 P fa95fcf2448c4166902c56d01332630c [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:52:35.578011 18766 raft_consensus.cc:2454] T 45e33723a3724e9f819ddd48688277f1 P fa95fcf2448c4166902c56d01332630c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0c75510608d24ff78855de400475c524 in term 1.
I20260809 22:52:35.578406 19203 raft_consensus.cc:2454] T 45e33723a3724e9f819ddd48688277f1 P 8e8fcf7e414b4304a18a79967f8a53ba [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0c75510608d24ff78855de400475c524 in term 1.
I20260809 22:52:35.578860 18867 leader_election.cc:302] T 45e33723a3724e9f819ddd48688277f1 P 0c75510608d24ff78855de400475c524 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0c75510608d24ff78855de400475c524, fa95fcf2448c4166902c56d01332630c; no voters:
I20260809 22:52:35.585043 19446 raft_consensus.cc:2781] T 45e33723a3724e9f819ddd48688277f1 P 0c75510608d24ff78855de400475c524 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 22:52:35.585289 19446 raft_consensus.cc:686] T 45e33723a3724e9f819ddd48688277f1 P 0c75510608d24ff78855de400475c524 [term 1 LEADER]: Becoming Leader. State: Replica: 0c75510608d24ff78855de400475c524, State: Running, Role: LEADER
I20260809 22:52:35.585701 19446 consensus_queue.cc:237] T 45e33723a3724e9f819ddd48688277f1 P 0c75510608d24ff78855de400475c524 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa95fcf2448c4166902c56d01332630c" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 35739 } } peers { permanent_uuid: "0c75510608d24ff78855de400475c524" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 34115 } } peers { permanent_uuid: "8e8fcf7e414b4304a18a79967f8a53ba" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 44931 } }
I20260809 22:52:35.588886 18513 catalog_manager.cc:5495] T 45e33723a3724e9f819ddd48688277f1 P 0c75510608d24ff78855de400475c524 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0c75510608d24ff78855de400475c524 (127.15.113.130). New cstate: current_term: 1 leader_uuid: "0c75510608d24ff78855de400475c524" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa95fcf2448c4166902c56d01332630c" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 35739 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0c75510608d24ff78855de400475c524" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 34115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8e8fcf7e414b4304a18a79967f8a53ba" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 44931 } health_report { overall_health: UNKNOWN } } }
I20260809 22:52:35.637279 19493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:52:35.629413 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38396 (local address 127.15.113.130:34115)
0809 22:52:35.630733 (+ 1320us) server_negotiation.cc:206] Beginning negotiation
0809 22:52:35.630738 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:52:35.630758 (+ 20us) server_negotiation.cc:407] Connection header received
0809 22:52:35.630807 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:52:35.630810 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:52:35.630870 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:52:35.630946 (+ 76us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 22:52:35.631500 (+ 554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:52:35.632726 (+ 1226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:52:35.634707 (+ 1981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:52:35.635644 (+ 937us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:52:35.635739 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:52:35.636293 (+ 554us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 22:52:35.636428 (+ 135us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 22:52:35.636496 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0809 22:52:35.636592 (+ 96us) server_negotiation.cc:299] Negotiation successful
0809 22:52:35.636655 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1119,"thread_start_us":839,"threads_started":1}
I20260809 22:52:35.647558 18766 raft_consensus.cc:1261] T 45e33723a3724e9f819ddd48688277f1 P fa95fcf2448c4166902c56d01332630c [term 1 FOLLOWER]: Refusing update from remote peer 0c75510608d24ff78855de400475c524: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 22:52:35.647495 19203 raft_consensus.cc:1261] T 45e33723a3724e9f819ddd48688277f1 P 8e8fcf7e414b4304a18a79967f8a53ba [term 1 FOLLOWER]: Refusing update from remote peer 0c75510608d24ff78855de400475c524: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 22:52:35.648355 19486 consensus_queue.cc:1035] T 45e33723a3724e9f819ddd48688277f1 P 0c75510608d24ff78855de400475c524 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e8fcf7e414b4304a18a79967f8a53ba" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 44931 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 22:52:35.648804 19446 consensus_queue.cc:1035] T 45e33723a3724e9f819ddd48688277f1 P 0c75510608d24ff78855de400475c524 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fa95fcf2448c4166902c56d01332630c" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 35739 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260809 22:52:35.669687 19422 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260809 22:52:37.255172 18514 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60654:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 22:52:37.255535 18514 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60654:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 22:52:37.260991 18514 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 72c0d959ee5c41298a2ea15ccc17437f: Sending DeleteTablet for 3 replicas of tablet 45e33723a3724e9f819ddd48688277f1
I20260809 22:52:37.261895 18947 tablet_service.cc:1514] Processing DeleteTablet for tablet 45e33723a3724e9f819ddd48688277f1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:52:37 UTC) from {username='kudu', principal='kudu/127.15.113.190@KRBTEST.COM'} at 127.0.0.1:38374
I20260809 22:52:37.261889 18729 tablet_service.cc:1514] Processing DeleteTablet for tablet 45e33723a3724e9f819ddd48688277f1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:52:37 UTC) from {username='kudu', principal='kudu/127.15.113.190@KRBTEST.COM'} at 127.0.0.1:34790
I20260809 22:52:37.261934 19161 tablet_service.cc:1514] Processing DeleteTablet for tablet 45e33723a3724e9f819ddd48688277f1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:52:37 UTC) from {username='kudu', principal='kudu/127.15.113.190@KRBTEST.COM'} at 127.0.0.1:48732
I20260809 22:52:37.262718 15814 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskItYPJR/build/asan/bin/kudu pid 18627
I20260809 22:52:37.275405 19554 tablet_replica.cc:330] T 45e33723a3724e9f819ddd48688277f1 P 0c75510608d24ff78855de400475c524: stopping tablet replica
I20260809 22:52:37.275805 19554 raft_consensus.cc:2229] T 45e33723a3724e9f819ddd48688277f1 P 0c75510608d24ff78855de400475c524 [term 1 LEADER]: Raft consensus shutting down.
I20260809 22:52:37.276582 19554 raft_consensus.cc:2258] T 45e33723a3724e9f819ddd48688277f1 P 0c75510608d24ff78855de400475c524 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 22:52:37.277885 19552 tablet_replica.cc:330] T 45e33723a3724e9f819ddd48688277f1 P 8e8fcf7e414b4304a18a79967f8a53ba: stopping tablet replica
I20260809 22:52:37.278142 19554 ts_tablet_manager.cc:1905] T 45e33723a3724e9f819ddd48688277f1 P 0c75510608d24ff78855de400475c524: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 22:52:37.278163 19552 raft_consensus.cc:2229] T 45e33723a3724e9f819ddd48688277f1 P 8e8fcf7e414b4304a18a79967f8a53ba [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 22:52:37.278494 19552 raft_consensus.cc:2258] T 45e33723a3724e9f819ddd48688277f1 P 8e8fcf7e414b4304a18a79967f8a53ba [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 22:52:37.280596 19552 ts_tablet_manager.cc:1905] T 45e33723a3724e9f819ddd48688277f1 P 8e8fcf7e414b4304a18a79967f8a53ba: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 22:52:37.286347 19554 ts_tablet_manager.cc:1918] T 45e33723a3724e9f819ddd48688277f1 P 0c75510608d24ff78855de400475c524: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260809 22:52:37.286602 19554 log.cc:1196] T 45e33723a3724e9f819ddd48688277f1 P 0c75510608d24ff78855de400475c524: Deleting WAL directory at /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315935617919-15814-0/minicluster-data/ts-1/wal/wals/45e33723a3724e9f819ddd48688277f1
I20260809 22:52:37.287493 19554 ts_tablet_manager.cc:1939] T 45e33723a3724e9f819ddd48688277f1 P 0c75510608d24ff78855de400475c524: Deleting consensus metadata
I20260809 22:52:37.288733 18491 catalog_manager.cc:4841] TS 0c75510608d24ff78855de400475c524 (127.15.113.130:34115): tablet 45e33723a3724e9f819ddd48688277f1 (table test-table [id=7252eb14b5be4af3bdda8e98c253c1e4]) successfully deleted
I20260809 22:52:37.289570 19550 tablet_replica.cc:330] T 45e33723a3724e9f819ddd48688277f1 P fa95fcf2448c4166902c56d01332630c: stopping tablet replica
I20260809 22:52:37.289680 19552 ts_tablet_manager.cc:1918] T 45e33723a3724e9f819ddd48688277f1 P 8e8fcf7e414b4304a18a79967f8a53ba: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260809 22:52:37.289809 19552 log.cc:1196] T 45e33723a3724e9f819ddd48688277f1 P 8e8fcf7e414b4304a18a79967f8a53ba: Deleting WAL directory at /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315935617919-15814-0/minicluster-data/ts-2/wal/wals/45e33723a3724e9f819ddd48688277f1
I20260809 22:52:37.289796 19550 raft_consensus.cc:2229] T 45e33723a3724e9f819ddd48688277f1 P fa95fcf2448c4166902c56d01332630c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 22:52:37.290102 19550 raft_consensus.cc:2258] T 45e33723a3724e9f819ddd48688277f1 P fa95fcf2448c4166902c56d01332630c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 22:52:37.290407 19552 ts_tablet_manager.cc:1939] T 45e33723a3724e9f819ddd48688277f1 P 8e8fcf7e414b4304a18a79967f8a53ba: Deleting consensus metadata
I20260809 22:52:37.291177 19550 ts_tablet_manager.cc:1905] T 45e33723a3724e9f819ddd48688277f1 P fa95fcf2448c4166902c56d01332630c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 22:52:37.291618 18487 catalog_manager.cc:4841] TS 8e8fcf7e414b4304a18a79967f8a53ba (127.15.113.131:44931): tablet 45e33723a3724e9f819ddd48688277f1 (table test-table [id=7252eb14b5be4af3bdda8e98c253c1e4]) successfully deleted
I20260809 22:52:37.298112 19550 ts_tablet_manager.cc:1918] T 45e33723a3724e9f819ddd48688277f1 P fa95fcf2448c4166902c56d01332630c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260809 22:52:37.298336 19550 log.cc:1196] T 45e33723a3724e9f819ddd48688277f1 P fa95fcf2448c4166902c56d01332630c: Deleting WAL directory at /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315935617919-15814-0/minicluster-data/ts-0/wal/wals/45e33723a3724e9f819ddd48688277f1
I20260809 22:52:37.298897 19550 ts_tablet_manager.cc:1939] T 45e33723a3724e9f819ddd48688277f1 P fa95fcf2448c4166902c56d01332630c: Deleting consensus metadata
Aug 09 22:52:37 dist-test-slave-ncp9 krb5kdc[18445](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315955, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.15.113.129@KRBTEST.COM
I20260809 22:52:37.299922 18491 catalog_manager.cc:4841] TS fa95fcf2448c4166902c56d01332630c (127.15.113.129:35739): tablet 45e33723a3724e9f819ddd48688277f1 (table test-table [id=7252eb14b5be4af3bdda8e98c253c1e4]) successfully deleted
I20260809 22:52:37.307833 19557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:52:37.264664 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34804 (local address 127.15.113.129:35739)
0809 22:52:37.289483 (+ 24819us) server_negotiation.cc:206] Beginning negotiation
0809 22:52:37.289487 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 22:52:37.289530 (+ 43us) server_negotiation.cc:407] Connection header received
0809 22:52:37.289592 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:52:37.289596 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:52:37.289706 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:52:37.290686 (+ 980us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:52:37.292332 (+ 1646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:52:37.293319 (+ 987us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:52:37.294821 (+ 1502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:52:37.295246 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:52:37.295387 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:52:37.301017 (+ 5630us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:52:37.301035 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:52:37.301053 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:52:37.301084 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:52:37.305003 (+ 3919us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:52:37.305640 (+ 637us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:52:37.305656 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:52:37.305659 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:52:37.305823 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:52:37.306188 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:52:37.306191 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:52:37.306193 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:52:37.306684 (+ 491us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:52:37.307052 (+ 368us) server_negotiation.cc:1035] Waiting for connection context
0809 22:52:37.307166 (+ 114us) server_negotiation.cc:299] Negotiation successful
0809 22:52:37.307374 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24579,"thread_start_us":24318,"threads_started":1}
W20260809 22:52:37.308641 18825 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34804
W20260809 22:52:37.309373 15814 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskItYPJR/build/asan/bin/kudu pid 18627: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 22:52:37.309674 15814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskItYPJR/build/asan/bin/kudu with pid 18627
I20260809 22:52:37.335294 15814 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskItYPJR/build/asan/bin/kudu pid 18837
Aug 09 22:52:37 dist-test-slave-ncp9 krb5kdc[18445](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315955, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.15.113.130@KRBTEST.COM
I20260809 22:52:37.351554 19558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:52:37.336288 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38398 (local address 127.15.113.130:34115)
0809 22:52:37.337085 (+ 797us) server_negotiation.cc:206] Beginning negotiation
0809 22:52:37.337089 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 22:52:37.337107 (+ 18us) server_negotiation.cc:407] Connection header received
0809 22:52:37.337168 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:52:37.337172 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:52:37.337239 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:52:37.337321 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:52:37.339072 (+ 1751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:52:37.340274 (+ 1202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:52:37.342074 (+ 1800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:52:37.342500 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:52:37.342591 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:52:37.346506 (+ 3915us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:52:37.346521 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:52:37.346537 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:52:37.346568 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:52:37.349373 (+ 2805us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:52:37.349950 (+ 577us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:52:37.349956 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:52:37.349959 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:52:37.350088 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:52:37.350349 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:52:37.350352 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:52:37.350354 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:52:37.350662 (+ 308us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:52:37.350889 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0809 22:52:37.351048 (+ 159us) server_negotiation.cc:299] Negotiation successful
0809 22:52:37.351218 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":640,"thread_start_us":436,"threads_started":1}
W20260809 22:52:37.352272 19046 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38398
W20260809 22:52:37.352921 15814 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskItYPJR/build/asan/bin/kudu pid 18837: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 22:52:37.353092 15814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskItYPJR/build/asan/bin/kudu with pid 18837
I20260809 22:52:37.388036 15814 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskItYPJR/build/asan/bin/kudu pid 19058
Aug 09 22:52:37 dist-test-slave-ncp9 krb5kdc[18445](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315955, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.15.113.131@KRBTEST.COM
I20260809 22:52:37.402119 19560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:52:37.388738 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48760 (local address 127.15.113.131:44931)
0809 22:52:37.389242 (+ 504us) server_negotiation.cc:206] Beginning negotiation
0809 22:52:37.389248 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:52:37.389265 (+ 17us) server_negotiation.cc:407] Connection header received
0809 22:52:37.389320 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:52:37.389323 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:52:37.389384 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:52:37.389476 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:52:37.390668 (+ 1192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:52:37.391417 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:52:37.392592 (+ 1175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:52:37.393018 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:52:37.393116 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:52:37.397138 (+ 4022us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:52:37.397154 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:52:37.397171 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:52:37.397202 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:52:37.399879 (+ 2677us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:52:37.400401 (+ 522us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:52:37.400407 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:52:37.400409 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:52:37.400526 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:52:37.400798 (+ 272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:52:37.400801 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:52:37.400803 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:52:37.401263 (+ 460us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:52:37.401480 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0809 22:52:37.401704 (+ 224us) server_negotiation.cc:299] Negotiation successful
0809 22:52:37.401864 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":383,"thread_start_us":261,"threads_started":1}
W20260809 22:52:37.402798 19270 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48760
W20260809 22:52:37.403379 15814 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskItYPJR/build/asan/bin/kudu pid 19058: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 22:52:37.403538 15814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskItYPJR/build/asan/bin/kudu with pid 19058
I20260809 22:52:37.434131 15814 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskItYPJR/build/asan/bin/kudu pid 18461
I20260809 22:52:37.435324 18583 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:52:37.551854 15814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskItYPJR/build/asan/bin/kudu with pid 18461
2026-08-09T22:52:37Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (8382 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315935617919-15814-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 22:52:37 dist-test-slave-ncp9 krb5kdc[19567](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 22:52:37 dist-test-slave-ncp9 krb5kdc[19567](info): set up 2 sockets
Aug 09 22:52:37 dist-test-slave-ncp9 krb5kdc[19567](info): commencing operation
krb5kdc: starting...
W20260809 22:52:38.754871 15814 mini_kdc.cc:121] Time spent starting KDC: real 1.153s user 0.004s sys 0.019s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315935617919-15814-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315935617919-15814-0/krb5kdc/test-user.keytab.
Aug 09 22:52:38 dist-test-slave-ncp9 krb5kdc[19567](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315958, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T22:52:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T22:52:38Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.113.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.113.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.113.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.190.keytab.
Entry for principal kudu/127.15.113.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.190.keytab.
I20260809 22:52:38.946437 15814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskItYPJR/build/asan/bin/kudu
/tmp/dist-test-taskItYPJR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315935617919-15814-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315935617919-15814-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315935617919-15814-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315935617919-15814-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.113.190:44033
--webserver_interface=127.15.113.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.190.keytab
--principal=kudu/127.15.113.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.113.148:38915
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.113.190:44033
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315935617919-15814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315935617919-15814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315935617919-15814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:52:39.263409 19589 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:52:39.263705 19589 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:52:39.263849 19589 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:52:39.270874 19589 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 22:52:39.270996 19589 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:52:39.271075 19589 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:52:39.271154 19589 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 22:52:39.271215 19589 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 22:52:39.280716 19589 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.113.148:38915
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315935617919-15814-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315935617919-15814-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.113.190:44033
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.190.keytab
--principal=kudu/127.15.113.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.113.190:44033
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315935617919-15814-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.113.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315935617919-15814-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:52:39.281926 19589 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:52:39.283221 19589 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:52:39.294368 19599 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:52:39.294339 19602 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:52:39.294647 19596 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:52:39.295149 19589 server_base.cc:1034] running on GCE node
I20260809 22:52:39.295845 19589 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:52:39.297173 19589 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:52:39.298368 19589 hybrid_clock.cc:648] HybridClock initialized: now 1786315959298324 us; error 32 us; skew 500 ppm
I20260809 22:52:39.298686 19589 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:52:39 dist-test-slave-ncp9 krb5kdc[19567](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315959, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.113.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:52:39.304073 19589 init.cc:377] Logged in from keytab as kudu/127.15.113.190@KRBTEST.COM (short username kudu)
I20260809 22:52:39.305460 19589 webserver.cc:460] Webserver started at http://127.15.113.190:34605/ using document root <none> and password file <none>
I20260809 22:52:39.306094 19589 fs_manager.cc:359] Metadata directory not provided
I20260809 22:52:39.306211 19589 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:52:39.306495 19589 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:52:39.309355 19589 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315935617919-15814-0/minicluster-data/master-0/data/instance:
uuid: "97122a4b909f46548ef51633ae71cbf6"
format_stamp: "Formatted at 2026-08-09 22:52:39 on dist-test-slave-ncp9"
server_key: "4169ab2fd000f5220c3a6ecdafb9ea00"
server_key_iv: "8b9b47af34bb8850b102362752103f3d"
server_key_version: "encryptionkey@0"
I20260809 22:52:39.310153 19589 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315935617919-15814-0/minicluster-data/master-0/wal/instance:
uuid: "97122a4b909f46548ef51633ae71cbf6"
format_stamp: "Formatted at 2026-08-09 22:52:39 on dist-test-slave-ncp9"
server_key: "4169ab2fd000f5220c3a6ecdafb9ea00"
server_key_iv: "8b9b47af34bb8850b102362752103f3d"
server_key_version: "encryptionkey@0"
I20260809 22:52:39.314949 19589 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260809 22:52:39.318157 19610 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:52:39.319317 19589 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 22:52:39.319465 19589 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315935617919-15814-0/minicluster-data/master-0/data,/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315935617919-15814-0/minicluster-data/master-0/wal
uuid: "97122a4b909f46548ef51633ae71cbf6"
format_stamp: "Formatted at 2026-08-09 22:52:39 on dist-test-slave-ncp9"
server_key: "4169ab2fd000f5220c3a6ecdafb9ea00"
server_key_iv: "8b9b47af34bb8850b102362752103f3d"
server_key_version: "encryptionkey@0"
I20260809 22:52:39.319635 19589 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315935617919-15814-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315935617919-15814-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315935617919-15814-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:52:39.335417 19589 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:52:39.338850 19589 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:52:39.339116 19589 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:52:39.355949 19589 rpc_server.cc:307] RPC server started. Bound to: 127.15.113.190:44033
I20260809 22:52:39.355966 19687 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.113.190:44033 every 8 connection(s)
I20260809 22:52:39.358489 19589 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315935617919-15814-0/minicluster-data/master-0/data/info.pb
I20260809 22:52:39.361980 15814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskItYPJR/build/asan/bin/kudu as pid 19589
I20260809 22:52:39.362254 15814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315935617919-15814-0/minicluster-data/master-0/wal/instance
I20260809 22:52:39.363049 15814 external_mini_cluster.cc:1428] Setting key 6b438105fa2adf08261044e78593c02a
I20260809 22:52:39.366226 19688 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:52:39.379994 19688 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 97122a4b909f46548ef51633ae71cbf6: Bootstrap starting.
Aug 09 22:52:39 dist-test-slave-ncp9 krb5kdc[19567](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315958, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.113.190@KRBTEST.COM
I20260809 22:52:39.383816 19688 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 97122a4b909f46548ef51633ae71cbf6: Neither blocks nor log segments found. Creating new log.
I20260809 22:52:39.385124 19688 log.cc:824] T 00000000000000000000000000000000 P 97122a4b909f46548ef51633ae71cbf6: Log is configured to *not* fsync() on all Append() calls
I20260809 22:52:39.388167 19688 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 97122a4b909f46548ef51633ae71cbf6: No bootstrap required, opened a new log
I20260809 22:52:39.392856 19697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:52:39.370125 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51718 (local address 127.15.113.190:44033)
0809 22:52:39.370974 (+ 849us) server_negotiation.cc:206] Beginning negotiation
0809 22:52:39.370988 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0809 22:52:39.371039 (+ 51us) server_negotiation.cc:407] Connection header received
0809 22:52:39.373245 (+ 2206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:52:39.373272 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:52:39.373661 (+ 389us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:52:39.373955 (+ 294us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:52:39.375958 (+ 2003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:52:39.377457 (+ 1499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:52:39.379215 (+ 1758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:52:39.379860 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:52:39.379991 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:52:39.384860 (+ 4869us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:52:39.384874 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:52:39.384912 (+ 38us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:52:39.384974 (+ 62us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:52:39.388383 (+ 3409us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:52:39.389237 (+ 854us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:52:39.389246 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:52:39.389255 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:52:39.389481 (+ 226us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:52:39.389925 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:52:39.389932 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:52:39.389934 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:52:39.390643 (+ 709us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:52:39.390874 (+ 231us) server_negotiation.cc:1035] Waiting for connection context
0809 22:52:39.391399 (+ 525us) server_negotiation.cc:299] Negotiation successful
0809 22:52:39.391791 (+ 392us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":611,"thread_start_us":378,"threads_started":1}
I20260809 22:52:39.397071 19688 raft_consensus.cc:348] T 00000000000000000000000000000000 P 97122a4b909f46548ef51633ae71cbf6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97122a4b909f46548ef51633ae71cbf6" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 44033 } }
I20260809 22:52:39.397436 19688 raft_consensus.cc:374] T 00000000000000000000000000000000 P 97122a4b909f46548ef51633ae71cbf6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:52:39.397538 19688 raft_consensus.cc:729] T 00000000000000000000000000000000 P 97122a4b909f46548ef51633ae71cbf6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97122a4b909f46548ef51633ae71cbf6, State: Initialized, Role: FOLLOWER
I20260809 22:52:39.398051 19688 consensus_queue.cc:260] T 00000000000000000000000000000000 P 97122a4b909f46548ef51633ae71cbf6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97122a4b909f46548ef51633ae71cbf6" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 44033 } }
I20260809 22:52:39.398270 19688 raft_consensus.cc:388] T 00000000000000000000000000000000 P 97122a4b909f46548ef51633ae71cbf6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 22:52:39.398396 19688 raft_consensus.cc:482] T 00000000000000000000000000000000 P 97122a4b909f46548ef51633ae71cbf6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 22:52:39.398551 19688 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 97122a4b909f46548ef51633ae71cbf6 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:52:39.413625 19688 raft_consensus.cc:504] T 00000000000000000000000000000000 P 97122a4b909f46548ef51633ae71cbf6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97122a4b909f46548ef51633ae71cbf6" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 44033 } }
I20260809 22:52:39.414276 19688 leader_election.cc:302] T 00000000000000000000000000000000 P 97122a4b909f46548ef51633ae71cbf6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 97122a4b909f46548ef51633ae71cbf6; no voters:
I20260809 22:52:39.414932 19688 leader_election.cc:288] T 00000000000000000000000000000000 P 97122a4b909f46548ef51633ae71cbf6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 22:52:39.415072 19701 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 97122a4b909f46548ef51633ae71cbf6 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 22:52:39.415756 19701 raft_consensus.cc:686] T 00000000000000000000000000000000 P 97122a4b909f46548ef51633ae71cbf6 [term 1 LEADER]: Becoming Leader. State: Replica: 97122a4b909f46548ef51633ae71cbf6, State: Running, Role: LEADER
I20260809 22:52:39.416374 19701 consensus_queue.cc:237] T 00000000000000000000000000000000 P 97122a4b909f46548ef51633ae71cbf6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97122a4b909f46548ef51633ae71cbf6" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 44033 } }
I20260809 22:52:39.416997 19688 sys_catalog.cc:564] T 00000000000000000000000000000000 P 97122a4b909f46548ef51633ae71cbf6 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 22:52:39.419497 19702 sys_catalog.cc:455] T 00000000000000000000000000000000 P 97122a4b909f46548ef51633ae71cbf6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "97122a4b909f46548ef51633ae71cbf6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97122a4b909f46548ef51633ae71cbf6" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 44033 } } }
I20260809 22:52:39.419857 19702 sys_catalog.cc:458] T 00000000000000000000000000000000 P 97122a4b909f46548ef51633ae71cbf6 [sys.catalog]: This master's current role is: LEADER
I20260809 22:52:39.420321 19705 sys_catalog.cc:455] T 00000000000000000000000000000000 P 97122a4b909f46548ef51633ae71cbf6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 97122a4b909f46548ef51633ae71cbf6. Latest consensus state: current_term: 1 leader_uuid: "97122a4b909f46548ef51633ae71cbf6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97122a4b909f46548ef51633ae71cbf6" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 44033 } } }
I20260809 22:52:39.420624 19705 sys_catalog.cc:458] T 00000000000000000000000000000000 P 97122a4b909f46548ef51633ae71cbf6 [sys.catalog]: This master's current role is: LEADER
I20260809 22:52:39.420913 19712 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 22:52:39.426780 19712 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 22:52:39.435485 19712 catalog_manager.cc:1347] Generated new cluster ID: cbef439eb15a410e9715906bab0035e9
I20260809 22:52:39.435587 19712 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 22:52:39.445058 19712 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 22:52:39.446776 19712 catalog_manager.cc:1504] Loading token signing keys...
I20260809 22:52:39.455971 19712 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 97122a4b909f46548ef51633ae71cbf6: Generated new TSK 0
I20260809 22:52:39.456862 19712 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.113.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.113.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.113.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.129.keytab.
Entry for principal kudu/127.15.113.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.129.keytab.
W20260809 22:52:40.275132 15814 mini_kdc.cc:262] Time spent creating service keytab for kudu/127.15.113.129: real 0.807s user 0.000s sys 0.016s
I20260809 22:52:40.276151 15814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskItYPJR/build/asan/bin/kudu
/tmp/dist-test-taskItYPJR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315935617919-15814-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315935617919-15814-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315935617919-15814-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315935617919-15814-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.113.129:0
--local_ip_for_outbound_sockets=127.15.113.129
--webserver_interface=127.15.113.129
--webserver_port=0
--tserver_master_addrs=127.15.113.190:44033
--keytab_file=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.129.keytab
--principal=kudu/127.15.113.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.113.148:38915
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315935617919-15814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315935617919-15814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315935617919-15814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:52:40.695781 19731 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:52:40.696178 19731 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:52:40.696364 19731 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:52:40.704124 19731 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:52:40.704255 19731 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:52:40.704391 19731 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.113.129
I20260809 22:52:40.714829 19731 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.113.148:38915
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315935617919-15814-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315935617919-15814-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.129.keytab
--principal=kudu/127.15.113.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.113.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315935617919-15814-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.113.129
--webserver_port=0
--tserver_master_addrs=127.15.113.190:44033
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.113.129
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315935617919-15814-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:52:40.716194 19731 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:52:40.717736 19731 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:52:40.731814 19739 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:52:40.731841 19737 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:52:40.731878 19736 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:52:40.732631 19731 server_base.cc:1034] running on GCE node
I20260809 22:52:40.733500 19731 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:52:40.734438 19731 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:52:40.735636 19731 hybrid_clock.cc:648] HybridClock initialized: now 1786315960735581 us; error 50 us; skew 500 ppm
I20260809 22:52:40.735966 19731 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:52:40 dist-test-slave-ncp9 krb5kdc[19567](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315960, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.113.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:52:40.739107 19731 init.cc:377] Logged in from keytab as kudu/127.15.113.129@KRBTEST.COM (short username kudu)
I20260809 22:52:40.740597 19731 webserver.cc:460] Webserver started at http://127.15.113.129:36683/ using document root <none> and password file <none>
I20260809 22:52:40.741582 19731 fs_manager.cc:359] Metadata directory not provided
I20260809 22:52:40.741714 19731 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:52:40.742022 19731 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:52:40.745070 19731 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315935617919-15814-0/minicluster-data/ts-0/data/instance:
uuid: "ac97fa26de4046a08fda316882e2174a"
format_stamp: "Formatted at 2026-08-09 22:52:40 on dist-test-slave-ncp9"
server_key: "57b63d28da2ff3d721d4b1ec5e36eede"
server_key_iv: "1ed583fb45ba8bedd945d2995008a96a"
server_key_version: "encryptionkey@0"
I20260809 22:52:40.745889 19731 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315935617919-15814-0/minicluster-data/ts-0/wal/instance:
uuid: "ac97fa26de4046a08fda316882e2174a"
format_stamp: "Formatted at 2026-08-09 22:52:40 on dist-test-slave-ncp9"
server_key: "57b63d28da2ff3d721d4b1ec5e36eede"
server_key_iv: "1ed583fb45ba8bedd945d2995008a96a"
server_key_version: "encryptionkey@0"
I20260809 22:52:40.751302 19731 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 22:52:40.754722 19749 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:52:40.756013 19731 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 22:52:40.756174 19731 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315935617919-15814-0/minicluster-data/ts-0/data,/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315935617919-15814-0/minicluster-data/ts-0/wal
uuid: "ac97fa26de4046a08fda316882e2174a"
format_stamp: "Formatted at 2026-08-09 22:52:40 on dist-test-slave-ncp9"
server_key: "57b63d28da2ff3d721d4b1ec5e36eede"
server_key_iv: "1ed583fb45ba8bedd945d2995008a96a"
server_key_version: "encryptionkey@0"
I20260809 22:52:40.756348 19731 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315935617919-15814-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315935617919-15814-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315935617919-15814-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:52:40.778558 19731 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:52:40.782295 19731 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:52:40.782536 19731 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:52:40.783555 19731 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:52:40.784799 19731 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:52:40.784878 19731 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:52:40.785019 19731 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:52:40.785087 19731 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:52:40.820271 19731 rpc_server.cc:307] RPC server started. Bound to: 127.15.113.129:42821
I20260809 22:52:40.820318 19940 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.113.129:42821 every 8 connection(s)
I20260809 22:52:40.821514 19731 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315935617919-15814-0/minicluster-data/ts-0/data/info.pb
I20260809 22:52:40.827159 15814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskItYPJR/build/asan/bin/kudu as pid 19731
I20260809 22:52:40.827368 15814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315935617919-15814-0/minicluster-data/ts-0/wal/instance
I20260809 22:52:40.828205 15814 external_mini_cluster.cc:1428] Setting key 7d9c1702f005d9fd0bfe9bc6741cc4f4
Aug 09 22:52:40 dist-test-slave-ncp9 krb5kdc[19567](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315960, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.113.129@KRBTEST.COM for kudu/127.15.113.190@KRBTEST.COM
I20260809 22:52:40.842831 19944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:52:40.824138 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.113.129:38961 (local address 127.15.113.190:44033)
0809 22:52:40.824635 (+ 497us) server_negotiation.cc:206] Beginning negotiation
0809 22:52:40.824639 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 22:52:40.825713 (+ 1074us) server_negotiation.cc:407] Connection header received
0809 22:52:40.827990 (+ 2277us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:52:40.827994 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:52:40.828061 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:52:40.828155 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:52:40.830175 (+ 2020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:52:40.830914 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:52:40.832454 (+ 1540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:52:40.833111 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:52:40.833218 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:52:40.836783 (+ 3565us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:52:40.836789 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:52:40.836792 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:52:40.836823 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:52:40.839709 (+ 2886us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:52:40.840372 (+ 663us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:52:40.840378 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:52:40.840382 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:52:40.840526 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:52:40.840925 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:52:40.840929 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:52:40.840932 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:52:40.841496 (+ 564us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:52:40.841661 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0809 22:52:40.842266 (+ 605us) server_negotiation.cc:299] Negotiation successful
0809 22:52:40.842473 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":378,"thread_start_us":243,"threads_started":1}
I20260809 22:52:40.843535 19948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:52:40.824810 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:44033 (local address 127.15.113.129:38961)
0809 22:52:40.825549 (+ 739us) negotiation.cc:107] Waiting for socket to connect
0809 22:52:40.825595 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0809 22:52:40.827688 (+ 2093us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:52:40.828435 (+ 747us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:52:40.828447 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:52:40.829183 (+ 736us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:52:40.829955 (+ 772us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:52:40.829989 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:52:40.831065 (+ 1076us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:52:40.831069 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:52:40.832318 (+ 1249us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:52:40.832331 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:52:40.833335 (+ 1004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:52:40.833343 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:52:40.833565 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:52:40.834165 (+ 600us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:52:40.834187 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:52:40.836531 (+ 2344us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:52:40.839840 (+ 3309us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:52:40.839847 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:52:40.839859 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:52:40.840259 (+ 400us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:52:40.840617 (+ 358us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:52:40.840629 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:52:40.840631 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:52:40.840805 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:52:40.841608 (+ 803us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:52:40.841614 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:52:40.841908 (+ 294us) client_negotiation.cc:769] Sending connection context
0809 22:52:40.842126 (+ 218us) client_negotiation.cc:240] Negotiation successful
0809 22:52:40.842496 (+ 370us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":503,"thread_start_us":260,"threads_started":1}
I20260809 22:52:40.844928 19941 heartbeater.cc:347] Connected to a master server at 127.15.113.190:44033
I20260809 22:52:40.845294 19941 heartbeater.cc:464] Registering TS with master...
I20260809 22:52:40.846117 19941 heartbeater.cc:511] Master 127.15.113.190:44033 requested a full tablet report, sending...
I20260809 22:52:40.848299 19634 ts_manager.cc:194] Registered new tserver with Master: ac97fa26de4046a08fda316882e2174a (127.15.113.129:42821)
I20260809 22:52:40.849740 19634 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.113.129@KRBTEST.COM'} at 127.15.113.129:38961
WARNING: no policy specified for kudu/127.15.113.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.113.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.113.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.130.keytab.
Entry for principal kudu/127.15.113.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.130.keytab.
W20260809 22:52:41.256219 15814 mini_kdc.cc:262] Time spent creating service keytab for kudu/127.15.113.130: real 0.425s user 0.002s sys 0.014s
I20260809 22:52:41.256982 15814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskItYPJR/build/asan/bin/kudu
/tmp/dist-test-taskItYPJR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315935617919-15814-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315935617919-15814-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315935617919-15814-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315935617919-15814-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.113.130:0
--local_ip_for_outbound_sockets=127.15.113.130
--webserver_interface=127.15.113.130
--webserver_port=0
--tserver_master_addrs=127.15.113.190:44033
--keytab_file=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.130.keytab
--principal=kudu/127.15.113.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.113.148:38915
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315935617919-15814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315935617919-15814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315935617919-15814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:52:41.602604 19953 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:52:41.602885 19953 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:52:41.603036 19953 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:52:41.610145 19953 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:52:41.610265 19953 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:52:41.610433 19953 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.113.130
I20260809 22:52:41.620067 19953 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.113.148:38915
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315935617919-15814-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315935617919-15814-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.130.keytab
--principal=kudu/127.15.113.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.113.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315935617919-15814-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.113.130
--webserver_port=0
--tserver_master_addrs=127.15.113.190:44033
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.113.130
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315935617919-15814-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:52:41.621320 19953 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:52:41.622682 19953 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:52:41.635890 19963 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:52:41.635964 19960 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:52:41.635980 19968 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:52:41.636999 19953 server_base.cc:1034] running on GCE node
I20260809 22:52:41.637583 19953 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:52:41.638489 19953 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:52:41.639720 19953 hybrid_clock.cc:648] HybridClock initialized: now 1786315961639662 us; error 46 us; skew 500 ppm
I20260809 22:52:41.640029 19953 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:52:41 dist-test-slave-ncp9 krb5kdc[19567](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315961, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.113.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:52:41.643103 19953 init.cc:377] Logged in from keytab as kudu/127.15.113.130@KRBTEST.COM (short username kudu)
I20260809 22:52:41.644527 19953 webserver.cc:460] Webserver started at http://127.15.113.130:33109/ using document root <none> and password file <none>
I20260809 22:52:41.645248 19953 fs_manager.cc:359] Metadata directory not provided
I20260809 22:52:41.645357 19953 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:52:41.645644 19953 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:52:41.648484 19953 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315935617919-15814-0/minicluster-data/ts-1/data/instance:
uuid: "be32ef7376af472685731f007a28d7e7"
format_stamp: "Formatted at 2026-08-09 22:52:41 on dist-test-slave-ncp9"
server_key: "f501c77fdb15a3bfe19325c647c101fd"
server_key_iv: "7965f7226b1fe32fc5726af131720fa1"
server_key_version: "encryptionkey@0"
I20260809 22:52:41.649302 19953 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315935617919-15814-0/minicluster-data/ts-1/wal/instance:
uuid: "be32ef7376af472685731f007a28d7e7"
format_stamp: "Formatted at 2026-08-09 22:52:41 on dist-test-slave-ncp9"
server_key: "f501c77fdb15a3bfe19325c647c101fd"
server_key_iv: "7965f7226b1fe32fc5726af131720fa1"
server_key_version: "encryptionkey@0"
I20260809 22:52:41.654430 19953 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260809 22:52:41.657796 19975 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:52:41.659060 19953 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 22:52:41.659214 19953 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315935617919-15814-0/minicluster-data/ts-1/data,/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315935617919-15814-0/minicluster-data/ts-1/wal
uuid: "be32ef7376af472685731f007a28d7e7"
format_stamp: "Formatted at 2026-08-09 22:52:41 on dist-test-slave-ncp9"
server_key: "f501c77fdb15a3bfe19325c647c101fd"
server_key_iv: "7965f7226b1fe32fc5726af131720fa1"
server_key_version: "encryptionkey@0"
I20260809 22:52:41.659389 19953 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315935617919-15814-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315935617919-15814-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315935617919-15814-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:52:41.676664 19953 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:52:41.680275 19953 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:52:41.680497 19953 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:52:41.681366 19953 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:52:41.682629 19953 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:52:41.682699 19953 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:52:41.682789 19953 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:52:41.682835 19953 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:52:41.713748 19953 rpc_server.cc:307] RPC server started. Bound to: 127.15.113.130:34915
I20260809 22:52:41.713799 20153 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.113.130:34915 every 8 connection(s)
I20260809 22:52:41.715034 19953 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315935617919-15814-0/minicluster-data/ts-1/data/info.pb
I20260809 22:52:41.716638 15814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskItYPJR/build/asan/bin/kudu as pid 19953
I20260809 22:52:41.716845 15814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315935617919-15814-0/minicluster-data/ts-1/wal/instance
I20260809 22:52:41.717374 15814 external_mini_cluster.cc:1428] Setting key df2bed55f13f8995cbb90fec6deb2bd7
Aug 09 22:52:41 dist-test-slave-ncp9 krb5kdc[19567](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315961, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.113.130@KRBTEST.COM for kudu/127.15.113.190@KRBTEST.COM
I20260809 22:52:41.735865 20161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:52:41.717993 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.113.130:58487 (local address 127.15.113.190:44033)
0809 22:52:41.718493 (+ 500us) server_negotiation.cc:206] Beginning negotiation
0809 22:52:41.718500 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:52:41.719703 (+ 1203us) server_negotiation.cc:407] Connection header received
0809 22:52:41.721678 (+ 1975us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:52:41.721684 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:52:41.721763 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:52:41.721869 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:52:41.723420 (+ 1551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:52:41.724540 (+ 1120us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:52:41.726108 (+ 1568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:52:41.726648 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:52:41.726767 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:52:41.730201 (+ 3434us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:52:41.730207 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:52:41.730210 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:52:41.730243 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:52:41.732743 (+ 2500us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:52:41.733492 (+ 749us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:52:41.733496 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:52:41.733499 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:52:41.733635 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:52:41.734067 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:52:41.734070 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:52:41.734072 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:52:41.734347 (+ 275us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:52:41.734482 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0809 22:52:41.735243 (+ 761us) server_negotiation.cc:299] Negotiation successful
0809 22:52:41.735481 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":381,"thread_start_us":251,"threads_started":1}
I20260809 22:52:41.736560 20162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:52:41.718668 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:44033 (local address 127.15.113.130:58487)
0809 22:52:41.719526 (+ 858us) negotiation.cc:107] Waiting for socket to connect
0809 22:52:41.719582 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0809 22:52:41.721398 (+ 1816us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:52:41.722069 (+ 671us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:52:41.722077 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:52:41.722659 (+ 582us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:52:41.723261 (+ 602us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:52:41.723281 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:52:41.724688 (+ 1407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:52:41.724692 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:52:41.725970 (+ 1278us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:52:41.725983 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:52:41.726841 (+ 858us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:52:41.726848 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:52:41.727124 (+ 276us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:52:41.727793 (+ 669us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:52:41.727820 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:52:41.729981 (+ 2161us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:52:41.732881 (+ 2900us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:52:41.732916 (+ 35us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:52:41.732933 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:52:41.733364 (+ 431us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:52:41.733744 (+ 380us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:52:41.733749 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:52:41.733753 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:52:41.733949 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:52:41.734478 (+ 529us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:52:41.734486 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:52:41.734820 (+ 334us) client_negotiation.cc:769] Sending connection context
0809 22:52:41.735090 (+ 270us) client_negotiation.cc:240] Negotiation successful
0809 22:52:41.735497 (+ 407us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":574,"thread_start_us":360,"threads_started":1}
I20260809 22:52:41.737919 20154 heartbeater.cc:347] Connected to a master server at 127.15.113.190:44033
I20260809 22:52:41.738204 20154 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.15.113.131@KRBTEST.COM; defaulting to no policy
I20260809 22:52:41.738897 20154 heartbeater.cc:511] Master 127.15.113.190:44033 requested a full tablet report, sending...
I20260809 22:52:41.740304 19634 ts_manager.cc:194] Registered new tserver with Master: be32ef7376af472685731f007a28d7e7 (127.15.113.130:34915)
I20260809 22:52:41.741155 19634 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.113.130@KRBTEST.COM'} at 127.15.113.130:58487
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.113.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.113.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.131.keytab.
Entry for principal kudu/127.15.113.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.131.keytab.
I20260809 22:52:41.763784 15814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskItYPJR/build/asan/bin/kudu
/tmp/dist-test-taskItYPJR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315935617919-15814-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315935617919-15814-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315935617919-15814-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315935617919-15814-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.113.131:0
--local_ip_for_outbound_sockets=127.15.113.131
--webserver_interface=127.15.113.131
--webserver_port=0
--tserver_master_addrs=127.15.113.190:44033
--keytab_file=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.131.keytab
--principal=kudu/127.15.113.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.113.148:38915
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315935617919-15814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315935617919-15814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315935617919-15814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 22:52:41.852716 19941 heartbeater.cc:503] Master 127.15.113.190:44033 was elected leader, sending a full tablet report...
W20260809 22:52:42.119764 20166 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:52:42.120031 20166 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:52:42.120152 20166 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:52:42.127158 20166 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:52:42.127265 20166 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:52:42.127411 20166 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.113.131
I20260809 22:52:42.136435 20166 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.113.148:38915
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315935617919-15814-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315935617919-15814-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.131.keytab
--principal=kudu/127.15.113.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.113.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315935617919-15814-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.113.131
--webserver_port=0
--tserver_master_addrs=127.15.113.190:44033
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.113.131
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315935617919-15814-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:52:42.137604 20166 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:52:42.138836 20166 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:52:42.150919 20177 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:52:42.150992 20179 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:52:42.150957 20175 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:52:42.151909 20166 server_base.cc:1034] running on GCE node
I20260809 22:52:42.152477 20166 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:52:42.153390 20166 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:52:42.154588 20166 hybrid_clock.cc:648] HybridClock initialized: now 1786315962154537 us; error 42 us; skew 500 ppm
I20260809 22:52:42.154883 20166 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:52:42 dist-test-slave-ncp9 krb5kdc[19567](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315962, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.113.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:52:42.157939 20166 init.cc:377] Logged in from keytab as kudu/127.15.113.131@KRBTEST.COM (short username kudu)
I20260809 22:52:42.159332 20166 webserver.cc:460] Webserver started at http://127.15.113.131:46741/ using document root <none> and password file <none>
I20260809 22:52:42.160024 20166 fs_manager.cc:359] Metadata directory not provided
I20260809 22:52:42.160149 20166 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:52:42.160439 20166 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:52:42.163444 20166 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315935617919-15814-0/minicluster-data/ts-2/data/instance:
uuid: "426acbfb29924b37a7bd930ea89d0972"
format_stamp: "Formatted at 2026-08-09 22:52:42 on dist-test-slave-ncp9"
server_key: "d15b16fef14bcbc81807bd92d7511647"
server_key_iv: "8f4e6f7d02d7af25bdba661134c60f3b"
server_key_version: "encryptionkey@0"
I20260809 22:52:42.164232 20166 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315935617919-15814-0/minicluster-data/ts-2/wal/instance:
uuid: "426acbfb29924b37a7bd930ea89d0972"
format_stamp: "Formatted at 2026-08-09 22:52:42 on dist-test-slave-ncp9"
server_key: "d15b16fef14bcbc81807bd92d7511647"
server_key_iv: "8f4e6f7d02d7af25bdba661134c60f3b"
server_key_version: "encryptionkey@0"
I20260809 22:52:42.169181 20166 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.008s sys 0.000s
I20260809 22:52:42.172437 20189 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:52:42.173738 20166 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 22:52:42.173884 20166 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315935617919-15814-0/minicluster-data/ts-2/data,/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315935617919-15814-0/minicluster-data/ts-2/wal
uuid: "426acbfb29924b37a7bd930ea89d0972"
format_stamp: "Formatted at 2026-08-09 22:52:42 on dist-test-slave-ncp9"
server_key: "d15b16fef14bcbc81807bd92d7511647"
server_key_iv: "8f4e6f7d02d7af25bdba661134c60f3b"
server_key_version: "encryptionkey@0"
I20260809 22:52:42.174064 20166 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315935617919-15814-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315935617919-15814-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315935617919-15814-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:52:42.217219 20166 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:52:42.221496 20166 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:52:42.221774 20166 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:52:42.222710 20166 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:52:42.224100 20166 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:52:42.224181 20166 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:52:42.224287 20166 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:52:42.224339 20166 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:52:42.253997 20166 rpc_server.cc:307] RPC server started. Bound to: 127.15.113.131:35919
I20260809 22:52:42.254045 20373 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.113.131:35919 every 8 connection(s)
I20260809 22:52:42.255213 20166 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315935617919-15814-0/minicluster-data/ts-2/data/info.pb
I20260809 22:52:42.262217 15814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskItYPJR/build/asan/bin/kudu as pid 20166
I20260809 22:52:42.262422 15814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315935617919-15814-0/minicluster-data/ts-2/wal/instance
I20260809 22:52:42.262965 15814 external_mini_cluster.cc:1428] Setting key fb713cd4db61e1e2322d97b8fd7b3c6d
Aug 09 22:52:42 dist-test-slave-ncp9 krb5kdc[19567](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315962, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.113.131@KRBTEST.COM for kudu/127.15.113.190@KRBTEST.COM
I20260809 22:52:42.274165 20377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:52:42.257605 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.113.131:35405 (local address 127.15.113.190:44033)
0809 22:52:42.258028 (+ 423us) server_negotiation.cc:206] Beginning negotiation
0809 22:52:42.258033 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:52:42.259006 (+ 973us) server_negotiation.cc:407] Connection header received
0809 22:52:42.260922 (+ 1916us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:52:42.260929 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:52:42.261003 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:52:42.261096 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:52:42.262758 (+ 1662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:52:42.263639 (+ 881us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:52:42.265311 (+ 1672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:52:42.265718 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:52:42.265825 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:52:42.268930 (+ 3105us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:52:42.268938 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:52:42.268943 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:52:42.268979 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:52:42.271299 (+ 2320us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:52:42.271958 (+ 659us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:52:42.271964 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:52:42.271968 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:52:42.272086 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:52:42.272487 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:52:42.272492 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:52:42.272496 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:52:42.272775 (+ 279us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:52:42.272951 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0809 22:52:42.273579 (+ 628us) server_negotiation.cc:299] Negotiation successful
0809 22:52:42.273808 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":318,"thread_start_us":198,"threads_started":1}
I20260809 22:52:42.275048 20378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:52:42.258194 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:44033 (local address 127.15.113.131:35405)
0809 22:52:42.258852 (+ 658us) negotiation.cc:107] Waiting for socket to connect
0809 22:52:42.258891 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0809 22:52:42.260598 (+ 1707us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:52:42.261303 (+ 705us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:52:42.261312 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:52:42.261787 (+ 475us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:52:42.262589 (+ 802us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:52:42.262618 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:52:42.263810 (+ 1192us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:52:42.263816 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:52:42.265175 (+ 1359us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:52:42.265189 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:52:42.265929 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:52:42.265937 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:52:42.266154 (+ 217us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:52:42.266711 (+ 557us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:52:42.266731 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:52:42.268704 (+ 1973us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:52:42.271429 (+ 2725us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:52:42.271439 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:52:42.271454 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:52:42.271846 (+ 392us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:52:42.272189 (+ 343us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:52:42.272193 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:52:42.272198 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:52:42.272376 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:52:42.272929 (+ 553us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:52:42.272936 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:52:42.273246 (+ 310us) client_negotiation.cc:769] Sending connection context
0809 22:52:42.273475 (+ 229us) client_negotiation.cc:240] Negotiation successful
0809 22:52:42.273954 (+ 479us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":438,"thread_start_us":253,"threads_started":1}
I20260809 22:52:42.276244 20374 heartbeater.cc:347] Connected to a master server at 127.15.113.190:44033
I20260809 22:52:42.276518 20374 heartbeater.cc:464] Registering TS with master...
I20260809 22:52:42.277351 20374 heartbeater.cc:511] Master 127.15.113.190:44033 requested a full tablet report, sending...
I20260809 22:52:42.278730 19634 ts_manager.cc:194] Registered new tserver with Master: 426acbfb29924b37a7bd930ea89d0972 (127.15.113.131:35919)
I20260809 22:52:42.279474 19634 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.113.131@KRBTEST.COM'} at 127.15.113.131:35405
I20260809 22:52:42.289556 15814 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260809 22:52:42.325260 20377 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:52:42.322742 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51728 (local address 127.15.113.190:44033)
0809 22:52:42.322915 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0809 22:52:42.322920 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:52:42.323427 (+ 507us) server_negotiation.cc:407] Connection header received
0809 22:52:42.323607 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:52:42.323611 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:52:42.323689 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:52:42.323801 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:52:42.325065 (+ 1264us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51728: BlockingRecv error: recv got EOF from 127.0.0.1:51728 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
I20260809 22:52:42.328433 15814 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskItYPJR/build/asan/bin/kudu pid 19731
W20260809 22:52:42.331009 15814 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskItYPJR/build/asan/bin/kudu pid 19731: Not authorized: Client connection negotiation failed: client connection to 127.15.113.129:42821: server requires authentication, but client does not have Kerberos credentials available
I20260809 22:52:42.331154 15814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskItYPJR/build/asan/bin/kudu with pid 19731
I20260809 22:52:42.353704 15814 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskItYPJR/build/asan/bin/kudu pid 19953
W20260809 22:52:42.356371 15814 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskItYPJR/build/asan/bin/kudu pid 19953: Not authorized: Client connection negotiation failed: client connection to 127.15.113.130:34915: server requires authentication, but client does not have Kerberos credentials available
I20260809 22:52:42.356570 15814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskItYPJR/build/asan/bin/kudu with pid 19953
I20260809 22:52:42.380697 15814 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskItYPJR/build/asan/bin/kudu pid 20166
W20260809 22:52:42.383603 15814 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskItYPJR/build/asan/bin/kudu pid 20166: Not authorized: Client connection negotiation failed: client connection to 127.15.113.131:35919: server requires authentication, but client does not have Kerberos credentials available
I20260809 22:52:42.383730 15814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskItYPJR/build/asan/bin/kudu with pid 20166
I20260809 22:52:42.402805 15814 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskItYPJR/build/asan/bin/kudu pid 19589
I20260809 22:52:42.403951 19681 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:52:42.522578 15814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskItYPJR/build/asan/bin/kudu with pid 19589
2026-08-09T22:52:42Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (4964 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786315935617919-15814-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 22:52:42 dist-test-slave-ncp9 krb5kdc[20402](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 22:52:42 dist-test-slave-ncp9 krb5kdc[20402](info): set up 2 sockets
Aug 09 22:52:42 dist-test-slave-ncp9 krb5kdc[20402](info): commencing operation
krb5kdc: starting...
W20260809 22:52:43.709573 15814 mini_kdc.cc:121] Time spent starting KDC: real 1.143s user 0.005s sys 0.019s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786315935617919-15814-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786315935617919-15814-0/krb5kdc/test-user.keytab.
Aug 09 22:52:43 dist-test-slave-ncp9 krb5kdc[20402](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315963, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T22:52:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T22:52:43Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.113.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.113.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.113.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.190.keytab.
Entry for principal kudu/127.15.113.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.190.keytab.
I20260809 22:52:43.899884 15814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskItYPJR/build/asan/bin/kudu
/tmp/dist-test-taskItYPJR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786315935617919-15814-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786315935617919-15814-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786315935617919-15814-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786315935617919-15814-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.113.190:46447
--webserver_interface=127.15.113.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.190.keytab
--principal=kudu/127.15.113.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.113.148:33273
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.113.190:46447
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786315935617919-15814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786315935617919-15814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786315935617919-15814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:52:44.215724 20421 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:52:44.215982 20421 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:52:44.216109 20421 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:52:44.223083 20421 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 22:52:44.223179 20421 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:52:44.223255 20421 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:52:44.223327 20421 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 22:52:44.223387 20421 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 22:52:44.232712 20421 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.113.148:33273
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786315935617919-15814-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786315935617919-15814-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.113.190:46447
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.190.keytab
--principal=kudu/127.15.113.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.113.190:46447
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786315935617919-15814-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.113.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786315935617919-15814-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:52:44.233906 20421 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:52:44.235148 20421 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:52:44.246052 20427 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:52:44.246135 20426 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:52:44.246078 20429 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:52:44.246842 20421 server_base.cc:1034] running on GCE node
I20260809 22:52:44.247567 20421 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:52:44.248806 20421 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:52:44.250007 20421 hybrid_clock.cc:648] HybridClock initialized: now 1786315964249965 us; error 33 us; skew 500 ppm
I20260809 22:52:44.250320 20421 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:52:44 dist-test-slave-ncp9 krb5kdc[20402](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315964, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.113.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:52:44.253218 20421 init.cc:377] Logged in from keytab as kudu/127.15.113.190@KRBTEST.COM (short username kudu)
I20260809 22:52:44.254568 20421 webserver.cc:460] Webserver started at http://127.15.113.190:39393/ using document root <none> and password file <none>
I20260809 22:52:44.255169 20421 fs_manager.cc:359] Metadata directory not provided
I20260809 22:52:44.255283 20421 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:52:44.255584 20421 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:52:44.258414 20421 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786315935617919-15814-0/minicluster-data/master-0/data/instance:
uuid: "d6879bc5b3634bd6a2c79d9236c53a29"
format_stamp: "Formatted at 2026-08-09 22:52:44 on dist-test-slave-ncp9"
server_key: "c69f919cacc690d47c96e74c1ed2c590"
server_key_iv: "212ea5781e1f81aa042be2bd5734602e"
server_key_version: "encryptionkey@0"
I20260809 22:52:44.259158 20421 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786315935617919-15814-0/minicluster-data/master-0/wal/instance:
uuid: "d6879bc5b3634bd6a2c79d9236c53a29"
format_stamp: "Formatted at 2026-08-09 22:52:44 on dist-test-slave-ncp9"
server_key: "c69f919cacc690d47c96e74c1ed2c590"
server_key_iv: "212ea5781e1f81aa042be2bd5734602e"
server_key_version: "encryptionkey@0"
I20260809 22:52:44.263881 20421 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260809 22:52:44.267050 20445 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:52:44.268234 20421 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260809 22:52:44.268374 20421 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786315935617919-15814-0/minicluster-data/master-0/data,/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786315935617919-15814-0/minicluster-data/master-0/wal
uuid: "d6879bc5b3634bd6a2c79d9236c53a29"
format_stamp: "Formatted at 2026-08-09 22:52:44 on dist-test-slave-ncp9"
server_key: "c69f919cacc690d47c96e74c1ed2c590"
server_key_iv: "212ea5781e1f81aa042be2bd5734602e"
server_key_version: "encryptionkey@0"
I20260809 22:52:44.268560 20421 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786315935617919-15814-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786315935617919-15814-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786315935617919-15814-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:52:44.281834 20421 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:52:44.285615 20421 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:52:44.285877 20421 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:52:44.302014 20421 rpc_server.cc:307] RPC server started. Bound to: 127.15.113.190:46447
I20260809 22:52:44.302047 20532 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.113.190:46447 every 8 connection(s)
I20260809 22:52:44.303238 20421 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786315935617919-15814-0/minicluster-data/master-0/data/info.pb
I20260809 22:52:44.305183 15814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskItYPJR/build/asan/bin/kudu as pid 20421
I20260809 22:52:44.305379 15814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786315935617919-15814-0/minicluster-data/master-0/wal/instance
I20260809 22:52:44.306003 15814 external_mini_cluster.cc:1428] Setting key ecb5bbb686ecbafe56bccd6634f8efba
I20260809 22:52:44.306856 20533 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:52:44.319454 20533 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d6879bc5b3634bd6a2c79d9236c53a29: Bootstrap starting.
Aug 09 22:52:44 dist-test-slave-ncp9 krb5kdc[20402](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315963, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.113.190@KRBTEST.COM
I20260809 22:52:44.322969 20533 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d6879bc5b3634bd6a2c79d9236c53a29: Neither blocks nor log segments found. Creating new log.
I20260809 22:52:44.323940 20533 log.cc:824] T 00000000000000000000000000000000 P d6879bc5b3634bd6a2c79d9236c53a29: Log is configured to *not* fsync() on all Append() calls
I20260809 22:52:44.326740 20533 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d6879bc5b3634bd6a2c79d9236c53a29: No bootstrap required, opened a new log
I20260809 22:52:44.330506 20537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:52:44.307674 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59714 (local address 127.15.113.190:46447)
0809 22:52:44.308420 (+ 746us) server_negotiation.cc:206] Beginning negotiation
0809 22:52:44.308430 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 22:52:44.308468 (+ 38us) server_negotiation.cc:407] Connection header received
0809 22:52:44.310117 (+ 1649us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:52:44.310138 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:52:44.310501 (+ 363us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:52:44.310823 (+ 322us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:52:44.312606 (+ 1783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:52:44.313957 (+ 1351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:52:44.315592 (+ 1635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:52:44.316203 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:52:44.316330 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:52:44.323663 (+ 7333us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:52:44.323670 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:52:44.323688 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:52:44.323721 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:52:44.326777 (+ 3056us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:52:44.327409 (+ 632us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:52:44.327415 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:52:44.327420 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:52:44.327580 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:52:44.327934 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:52:44.327938 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:52:44.327940 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:52:44.328448 (+ 508us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:52:44.328733 (+ 285us) server_negotiation.cc:1035] Waiting for connection context
0809 22:52:44.329191 (+ 458us) server_negotiation.cc:299] Negotiation successful
0809 22:52:44.329562 (+ 371us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":550,"thread_start_us":345,"threads_started":1}
I20260809 22:52:44.333973 15814 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 22:52:44.334227 20533 raft_consensus.cc:348] T 00000000000000000000000000000000 P d6879bc5b3634bd6a2c79d9236c53a29 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d6879bc5b3634bd6a2c79d9236c53a29" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 46447 } }
I20260809 22:52:44.334544 20533 raft_consensus.cc:374] T 00000000000000000000000000000000 P d6879bc5b3634bd6a2c79d9236c53a29 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:52:44.334630 20533 raft_consensus.cc:729] T 00000000000000000000000000000000 P d6879bc5b3634bd6a2c79d9236c53a29 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6879bc5b3634bd6a2c79d9236c53a29, State: Initialized, Role: FOLLOWER
I20260809 22:52:44.335152 20533 consensus_queue.cc:260] T 00000000000000000000000000000000 P d6879bc5b3634bd6a2c79d9236c53a29 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d6879bc5b3634bd6a2c79d9236c53a29" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 46447 } }
I20260809 22:52:44.335368 20533 raft_consensus.cc:388] T 00000000000000000000000000000000 P d6879bc5b3634bd6a2c79d9236c53a29 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 22:52:44.335501 20533 raft_consensus.cc:482] T 00000000000000000000000000000000 P d6879bc5b3634bd6a2c79d9236c53a29 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 22:52:44.335642 20533 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d6879bc5b3634bd6a2c79d9236c53a29 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:52:44.337697 20533 raft_consensus.cc:504] T 00000000000000000000000000000000 P d6879bc5b3634bd6a2c79d9236c53a29 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d6879bc5b3634bd6a2c79d9236c53a29" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 46447 } }
I20260809 22:52:44.338127 20533 leader_election.cc:302] T 00000000000000000000000000000000 P d6879bc5b3634bd6a2c79d9236c53a29 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d6879bc5b3634bd6a2c79d9236c53a29; no voters:
I20260809 22:52:44.338699 20533 leader_election.cc:288] T 00000000000000000000000000000000 P d6879bc5b3634bd6a2c79d9236c53a29 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 22:52:44.338790 20539 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d6879bc5b3634bd6a2c79d9236c53a29 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 22:52:44.340792 20533 sys_catalog.cc:564] T 00000000000000000000000000000000 P d6879bc5b3634bd6a2c79d9236c53a29 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 22:52:44.343303 20539 raft_consensus.cc:686] T 00000000000000000000000000000000 P d6879bc5b3634bd6a2c79d9236c53a29 [term 1 LEADER]: Becoming Leader. State: Replica: d6879bc5b3634bd6a2c79d9236c53a29, State: Running, Role: LEADER
I20260809 22:52:44.343964 20539 consensus_queue.cc:237] T 00000000000000000000000000000000 P d6879bc5b3634bd6a2c79d9236c53a29 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d6879bc5b3634bd6a2c79d9236c53a29" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 46447 } }
I20260809 22:52:44.346802 20542 sys_catalog.cc:455] T 00000000000000000000000000000000 P d6879bc5b3634bd6a2c79d9236c53a29 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d6879bc5b3634bd6a2c79d9236c53a29. Latest consensus state: current_term: 1 leader_uuid: "d6879bc5b3634bd6a2c79d9236c53a29" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d6879bc5b3634bd6a2c79d9236c53a29" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 46447 } } }
I20260809 22:52:44.347172 20542 sys_catalog.cc:458] T 00000000000000000000000000000000 P d6879bc5b3634bd6a2c79d9236c53a29 [sys.catalog]: This master's current role is: LEADER
I20260809 22:52:44.347774 20541 sys_catalog.cc:455] T 00000000000000000000000000000000 P d6879bc5b3634bd6a2c79d9236c53a29 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d6879bc5b3634bd6a2c79d9236c53a29" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d6879bc5b3634bd6a2c79d9236c53a29" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 46447 } } }
I20260809 22:52:44.348061 20541 sys_catalog.cc:458] T 00000000000000000000000000000000 P d6879bc5b3634bd6a2c79d9236c53a29 [sys.catalog]: This master's current role is: LEADER
I20260809 22:52:44.354866 20553 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 22:52:44.360355 20553 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 22:52:44.367010 20573 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260809 22:52:44.369570 20553 catalog_manager.cc:1347] Generated new cluster ID: a03708329a1b47089fc787bf64e12a19
I20260809 22:52:44.369746 20553 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 22:52:44.380653 20553 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 22:52:44.382025 20553 catalog_manager.cc:1504] Loading token signing keys...
I20260809 22:52:44.385202 20537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:52:44.376711 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59720 (local address 127.15.113.190:46447)
0809 22:52:44.376871 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0809 22:52:44.376877 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:52:44.377139 (+ 262us) server_negotiation.cc:407] Connection header received
0809 22:52:44.377340 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:52:44.377345 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:52:44.377429 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:52:44.377548 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:52:44.378786 (+ 1238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:52:44.379771 (+ 985us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:52:44.381401 (+ 1630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:52:44.382095 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:52:44.382238 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:52:44.383631 (+ 1393us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:52:44.383638 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:52:44.383643 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 22:52:44.384360 (+ 717us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:52:44.384417 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 22:52:44.384442 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:52:44.384534 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0809 22:52:44.384702 (+ 168us) server_negotiation.cc:299] Negotiation successful
0809 22:52:44.384820 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260809 22:52:44.397428 20553 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d6879bc5b3634bd6a2c79d9236c53a29: Generated new TSK 0
I20260809 22:52:44.398128 20553 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 22:52:44.405467 15814 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskItYPJR/build/asan/bin/kudu pid 20421
I20260809 22:52:44.406738 20523 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:52:44.493554 15814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskItYPJR/build/asan/bin/kudu with pid 20421
2026-08-09T22:52:44Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1966 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315935617919-15814-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 22:52:44 dist-test-slave-ncp9 krb5kdc[20585](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 22:52:44 dist-test-slave-ncp9 krb5kdc[20585](info): set up 2 sockets
Aug 09 22:52:44 dist-test-slave-ncp9 krb5kdc[20585](info): commencing operation
krb5kdc: starting...
W20260809 22:52:45.663589 15814 mini_kdc.cc:121] Time spent starting KDC: real 1.131s user 0.002s sys 0.021s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315935617919-15814-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315935617919-15814-0/krb5kdc/test-user.keytab.
Aug 09 22:52:45 dist-test-slave-ncp9 krb5kdc[20585](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315965, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T22:52:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T22:52:45Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.113.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.113.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.113.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.190.keytab.
Entry for principal kudu/127.15.113.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.190.keytab.
I20260809 22:52:45.860255 15814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskItYPJR/build/asan/bin/kudu
/tmp/dist-test-taskItYPJR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315935617919-15814-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315935617919-15814-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315935617919-15814-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315935617919-15814-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.113.190:34261
--webserver_interface=127.15.113.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.190.keytab
--principal=kudu/127.15.113.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.113.148:43887
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.113.190:34261
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315935617919-15814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315935617919-15814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315935617919-15814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:52:46.173617 20613 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:52:46.173885 20613 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:52:46.174006 20613 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:52:46.180852 20613 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 22:52:46.180965 20613 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:52:46.181041 20613 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:52:46.181113 20613 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 22:52:46.181176 20613 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 22:52:46.190501 20613 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.113.148:43887
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315935617919-15814-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315935617919-15814-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.113.190:34261
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.190.keytab
--principal=kudu/127.15.113.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.113.190:34261
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315935617919-15814-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.113.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315935617919-15814-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:52:46.191684 20613 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:52:46.192965 20613 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:52:46.203999 20621 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:52:46.204087 20622 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:52:46.204048 20625 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:52:46.204921 20613 server_base.cc:1034] running on GCE node
I20260809 22:52:46.205648 20613 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:52:46.206920 20613 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:52:46.208148 20613 hybrid_clock.cc:648] HybridClock initialized: now 1786315966208092 us; error 40 us; skew 500 ppm
I20260809 22:52:46.208482 20613 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:52:46 dist-test-slave-ncp9 krb5kdc[20585](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315966, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.113.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:52:46.211462 20613 init.cc:377] Logged in from keytab as kudu/127.15.113.190@KRBTEST.COM (short username kudu)
I20260809 22:52:46.212821 20613 webserver.cc:460] Webserver started at http://127.15.113.190:42719/ using document root <none> and password file <none>
I20260809 22:52:46.213459 20613 fs_manager.cc:359] Metadata directory not provided
I20260809 22:52:46.213586 20613 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:52:46.213881 20613 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:52:46.216696 20613 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315935617919-15814-0/minicluster-data/master-0/data/instance:
uuid: "53a032f43b8145c18569239e79ceb27d"
format_stamp: "Formatted at 2026-08-09 22:52:46 on dist-test-slave-ncp9"
server_key: "3d97bd692e3254e28909c9fddaed6233"
server_key_iv: "767b970cb8cd4acb955cbf48de01ff70"
server_key_version: "encryptionkey@0"
I20260809 22:52:46.217469 20613 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315935617919-15814-0/minicluster-data/master-0/wal/instance:
uuid: "53a032f43b8145c18569239e79ceb27d"
format_stamp: "Formatted at 2026-08-09 22:52:46 on dist-test-slave-ncp9"
server_key: "3d97bd692e3254e28909c9fddaed6233"
server_key_iv: "767b970cb8cd4acb955cbf48de01ff70"
server_key_version: "encryptionkey@0"
I20260809 22:52:46.222270 20613 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.004s
I20260809 22:52:46.225487 20632 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:52:46.226717 20613 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 22:52:46.226871 20613 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315935617919-15814-0/minicluster-data/master-0/data,/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315935617919-15814-0/minicluster-data/master-0/wal
uuid: "53a032f43b8145c18569239e79ceb27d"
format_stamp: "Formatted at 2026-08-09 22:52:46 on dist-test-slave-ncp9"
server_key: "3d97bd692e3254e28909c9fddaed6233"
server_key_iv: "767b970cb8cd4acb955cbf48de01ff70"
server_key_version: "encryptionkey@0"
I20260809 22:52:46.227038 20613 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315935617919-15814-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315935617919-15814-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315935617919-15814-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:52:46.243856 20613 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:52:46.247810 20613 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:52:46.248064 20613 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:52:46.264390 20613 rpc_server.cc:307] RPC server started. Bound to: 127.15.113.190:34261
I20260809 22:52:46.264429 20718 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.113.190:34261 every 8 connection(s)
I20260809 22:52:46.265635 20613 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315935617919-15814-0/minicluster-data/master-0/data/info.pb
I20260809 22:52:46.266121 15814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskItYPJR/build/asan/bin/kudu as pid 20613
I20260809 22:52:46.266352 15814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315935617919-15814-0/minicluster-data/master-0/wal/instance
I20260809 22:52:46.267194 15814 external_mini_cluster.cc:1428] Setting key 17bd974304187ec8a323e3d7f0c74819
I20260809 22:52:46.269897 20719 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 09 22:52:46 dist-test-slave-ncp9 krb5kdc[20585](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315965, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.113.190@KRBTEST.COM
I20260809 22:52:46.283420 20719 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 53a032f43b8145c18569239e79ceb27d: Bootstrap starting.
I20260809 22:52:46.286729 20719 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 53a032f43b8145c18569239e79ceb27d: Neither blocks nor log segments found. Creating new log.
I20260809 22:52:46.287911 20719 log.cc:824] T 00000000000000000000000000000000 P 53a032f43b8145c18569239e79ceb27d: Log is configured to *not* fsync() on all Append() calls
I20260809 22:52:46.290910 20719 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 53a032f43b8145c18569239e79ceb27d: No bootstrap required, opened a new log
I20260809 22:52:46.291286 20723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:52:46.269249 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52654 (local address 127.15.113.190:34261)
0809 22:52:46.270577 (+ 1328us) server_negotiation.cc:206] Beginning negotiation
0809 22:52:46.270592 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0809 22:52:46.270709 (+ 117us) server_negotiation.cc:407] Connection header received
0809 22:52:46.273100 (+ 2391us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:52:46.273129 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:52:46.273772 (+ 643us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:52:46.274215 (+ 443us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:52:46.275415 (+ 1200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:52:46.277880 (+ 2465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:52:46.279051 (+ 1171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:52:46.280035 (+ 984us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:52:46.280226 (+ 191us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:52:46.284292 (+ 4066us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:52:46.284300 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:52:46.284314 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:52:46.284348 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:52:46.287312 (+ 2964us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:52:46.288066 (+ 754us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:52:46.288074 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:52:46.288082 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:52:46.288242 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:52:46.288672 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:52:46.288677 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:52:46.288680 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:52:46.289338 (+ 658us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:52:46.289557 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0809 22:52:46.289970 (+ 413us) server_negotiation.cc:299] Negotiation successful
0809 22:52:46.290288 (+ 318us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1110,"thread_start_us":871,"threads_started":1}
I20260809 22:52:46.298465 20719 raft_consensus.cc:348] T 00000000000000000000000000000000 P 53a032f43b8145c18569239e79ceb27d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53a032f43b8145c18569239e79ceb27d" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 34261 } }
I20260809 22:52:46.298928 20719 raft_consensus.cc:374] T 00000000000000000000000000000000 P 53a032f43b8145c18569239e79ceb27d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:52:46.299036 20719 raft_consensus.cc:729] T 00000000000000000000000000000000 P 53a032f43b8145c18569239e79ceb27d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 53a032f43b8145c18569239e79ceb27d, State: Initialized, Role: FOLLOWER
I20260809 22:52:46.299580 20719 consensus_queue.cc:260] T 00000000000000000000000000000000 P 53a032f43b8145c18569239e79ceb27d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53a032f43b8145c18569239e79ceb27d" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 34261 } }
I20260809 22:52:46.299789 20719 raft_consensus.cc:388] T 00000000000000000000000000000000 P 53a032f43b8145c18569239e79ceb27d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 22:52:46.299885 20719 raft_consensus.cc:482] T 00000000000000000000000000000000 P 53a032f43b8145c18569239e79ceb27d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 22:52:46.300000 20719 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 53a032f43b8145c18569239e79ceb27d [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:52:46.301975 20719 raft_consensus.cc:504] T 00000000000000000000000000000000 P 53a032f43b8145c18569239e79ceb27d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53a032f43b8145c18569239e79ceb27d" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 34261 } }
I20260809 22:52:46.302402 20719 leader_election.cc:302] T 00000000000000000000000000000000 P 53a032f43b8145c18569239e79ceb27d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 53a032f43b8145c18569239e79ceb27d; no voters:
I20260809 22:52:46.302994 20719 leader_election.cc:288] T 00000000000000000000000000000000 P 53a032f43b8145c18569239e79ceb27d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 22:52:46.303110 20725 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 53a032f43b8145c18569239e79ceb27d [term 1 FOLLOWER]: Leader election won for term 1
I20260809 22:52:46.303866 20725 raft_consensus.cc:686] T 00000000000000000000000000000000 P 53a032f43b8145c18569239e79ceb27d [term 1 LEADER]: Becoming Leader. State: Replica: 53a032f43b8145c18569239e79ceb27d, State: Running, Role: LEADER
I20260809 22:52:46.304339 20725 consensus_queue.cc:237] T 00000000000000000000000000000000 P 53a032f43b8145c18569239e79ceb27d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53a032f43b8145c18569239e79ceb27d" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 34261 } }
I20260809 22:52:46.304981 20719 sys_catalog.cc:564] T 00000000000000000000000000000000 P 53a032f43b8145c18569239e79ceb27d [sys.catalog]: configured and running, proceeding with master startup.
I20260809 22:52:46.307387 20728 sys_catalog.cc:455] T 00000000000000000000000000000000 P 53a032f43b8145c18569239e79ceb27d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 53a032f43b8145c18569239e79ceb27d. Latest consensus state: current_term: 1 leader_uuid: "53a032f43b8145c18569239e79ceb27d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53a032f43b8145c18569239e79ceb27d" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 34261 } } }
I20260809 22:52:46.307399 20726 sys_catalog.cc:455] T 00000000000000000000000000000000 P 53a032f43b8145c18569239e79ceb27d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "53a032f43b8145c18569239e79ceb27d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53a032f43b8145c18569239e79ceb27d" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 34261 } } }
I20260809 22:52:46.307829 20726 sys_catalog.cc:458] T 00000000000000000000000000000000 P 53a032f43b8145c18569239e79ceb27d [sys.catalog]: This master's current role is: LEADER
I20260809 22:52:46.311381 20728 sys_catalog.cc:458] T 00000000000000000000000000000000 P 53a032f43b8145c18569239e79ceb27d [sys.catalog]: This master's current role is: LEADER
I20260809 22:52:46.311573 20738 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 22:52:46.316355 20738 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 22:52:46.324215 20738 catalog_manager.cc:1347] Generated new cluster ID: 85d6c5fff9f74e479990d2427ae5697c
I20260809 22:52:46.324312 20738 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 22:52:46.335893 20738 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 22:52:46.337059 20738 catalog_manager.cc:1504] Loading token signing keys...
I20260809 22:52:46.351884 20738 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 53a032f43b8145c18569239e79ceb27d: Generated new TSK 0
I20260809 22:52:46.352936 20738 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.113.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.113.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.113.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.129.keytab.
Entry for principal kudu/127.15.113.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.129.keytab.
I20260809 22:52:46.410694 15814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskItYPJR/build/asan/bin/kudu
/tmp/dist-test-taskItYPJR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315935617919-15814-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315935617919-15814-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315935617919-15814-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315935617919-15814-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.113.129:0
--local_ip_for_outbound_sockets=127.15.113.129
--webserver_interface=127.15.113.129
--webserver_port=0
--tserver_master_addrs=127.15.113.190:34261
--keytab_file=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.129.keytab
--principal=kudu/127.15.113.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.113.148:43887
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315935617919-15814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315935617919-15814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315935617919-15814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:52:46.733620 20755 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:52:46.733911 20755 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:52:46.734059 20755 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:52:46.741438 20755 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:52:46.741570 20755 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:52:46.741734 20755 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.113.129
I20260809 22:52:46.751487 20755 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.113.148:43887
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315935617919-15814-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315935617919-15814-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.129.keytab
--principal=kudu/127.15.113.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.113.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315935617919-15814-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.113.129
--webserver_port=0
--tserver_master_addrs=127.15.113.190:34261
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.113.129
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315935617919-15814-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:52:46.752737 20755 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:52:46.754070 20755 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:52:46.766831 20763 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:52:46.766963 20767 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:52:46.767138 20765 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:52:46.768373 20755 server_base.cc:1034] running on GCE node
I20260809 22:52:46.769078 20755 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:52:46.769989 20755 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:52:46.771214 20755 hybrid_clock.cc:648] HybridClock initialized: now 1786315966771146 us; error 52 us; skew 500 ppm
I20260809 22:52:46.771556 20755 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:52:46 dist-test-slave-ncp9 krb5kdc[20585](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315966, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.113.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:52:46.774600 20755 init.cc:377] Logged in from keytab as kudu/127.15.113.129@KRBTEST.COM (short username kudu)
I20260809 22:52:46.775919 20755 webserver.cc:460] Webserver started at http://127.15.113.129:34579/ using document root <none> and password file <none>
I20260809 22:52:46.776566 20755 fs_manager.cc:359] Metadata directory not provided
I20260809 22:52:46.776687 20755 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:52:46.777019 20755 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:52:46.779816 20755 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315935617919-15814-0/minicluster-data/ts-0/data/instance:
uuid: "d55d0555bc8f4f4d980ec1dfe4bfa1ff"
format_stamp: "Formatted at 2026-08-09 22:52:46 on dist-test-slave-ncp9"
server_key: "d9a9796c3dfe1fd2ba1164af5f33c72f"
server_key_iv: "00af8082b2aab5895e0786e87d413817"
server_key_version: "encryptionkey@0"
I20260809 22:52:46.780555 20755 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315935617919-15814-0/minicluster-data/ts-0/wal/instance:
uuid: "d55d0555bc8f4f4d980ec1dfe4bfa1ff"
format_stamp: "Formatted at 2026-08-09 22:52:46 on dist-test-slave-ncp9"
server_key: "d9a9796c3dfe1fd2ba1164af5f33c72f"
server_key_iv: "00af8082b2aab5895e0786e87d413817"
server_key_version: "encryptionkey@0"
I20260809 22:52:46.785378 20755 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260809 22:52:46.788599 20776 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:52:46.789809 20755 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 22:52:46.789953 20755 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315935617919-15814-0/minicluster-data/ts-0/data,/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315935617919-15814-0/minicluster-data/ts-0/wal
uuid: "d55d0555bc8f4f4d980ec1dfe4bfa1ff"
format_stamp: "Formatted at 2026-08-09 22:52:46 on dist-test-slave-ncp9"
server_key: "d9a9796c3dfe1fd2ba1164af5f33c72f"
server_key_iv: "00af8082b2aab5895e0786e87d413817"
server_key_version: "encryptionkey@0"
I20260809 22:52:46.790123 20755 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315935617919-15814-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315935617919-15814-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315935617919-15814-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:52:46.809226 20755 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:52:46.812855 20755 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:52:46.813140 20755 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:52:46.814006 20755 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:52:46.815269 20755 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:52:46.815346 20755 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:52:46.815460 20755 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:52:46.815523 20755 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:52:46.846053 20755 rpc_server.cc:307] RPC server started. Bound to: 127.15.113.129:42369
I20260809 22:52:46.846117 20958 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.113.129:42369 every 8 connection(s)
I20260809 22:52:46.847258 20755 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315935617919-15814-0/minicluster-data/ts-0/data/info.pb
I20260809 22:52:46.850030 15814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskItYPJR/build/asan/bin/kudu as pid 20755
I20260809 22:52:46.850265 15814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315935617919-15814-0/minicluster-data/ts-0/wal/instance
I20260809 22:52:46.851120 15814 external_mini_cluster.cc:1428] Setting key f383534617d435f8903b4e857519ed05
Aug 09 22:52:46 dist-test-slave-ncp9 krb5kdc[20585](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315966, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.113.129@KRBTEST.COM for kudu/127.15.113.190@KRBTEST.COM
I20260809 22:52:46.866353 20963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:52:46.849706 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.113.129:44523 (local address 127.15.113.190:34261)
0809 22:52:46.850127 (+ 421us) server_negotiation.cc:206] Beginning negotiation
0809 22:52:46.850133 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:52:46.851342 (+ 1209us) server_negotiation.cc:407] Connection header received
0809 22:52:46.853223 (+ 1881us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:52:46.853228 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:52:46.853304 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:52:46.853397 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:52:46.854939 (+ 1542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:52:46.855631 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:52:46.857185 (+ 1554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:52:46.857601 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:52:46.857708 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:52:46.860992 (+ 3284us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:52:46.860997 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:52:46.861000 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:52:46.861030 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:52:46.863361 (+ 2331us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:52:46.864225 (+ 864us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:52:46.864229 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:52:46.864231 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:52:46.864336 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:52:46.864755 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:52:46.864758 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:52:46.864760 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:52:46.865048 (+ 288us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:52:46.865164 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0809 22:52:46.865942 (+ 778us) server_negotiation.cc:299] Negotiation successful
0809 22:52:46.866092 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":316,"thread_start_us":212,"threads_started":1}
I20260809 22:52:46.867658 20964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:52:46.850394 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:34261 (local address 127.15.113.129:44523)
0809 22:52:46.851168 (+ 774us) negotiation.cc:107] Waiting for socket to connect
0809 22:52:46.851225 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0809 22:52:46.852968 (+ 1743us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:52:46.853631 (+ 663us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:52:46.853639 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:52:46.854141 (+ 502us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:52:46.854781 (+ 640us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:52:46.854800 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:52:46.855772 (+ 972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:52:46.855776 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:52:46.857046 (+ 1270us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:52:46.857059 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:52:46.857812 (+ 753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:52:46.857819 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:52:46.858045 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:52:46.858656 (+ 611us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:52:46.858677 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:52:46.860754 (+ 2077us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:52:46.863494 (+ 2740us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:52:46.863502 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:52:46.863516 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:52:46.864127 (+ 611us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:52:46.864427 (+ 300us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:52:46.864430 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:52:46.864433 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:52:46.864665 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:52:46.865172 (+ 507us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:52:46.865179 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:52:46.865545 (+ 366us) client_negotiation.cc:769] Sending connection context
0809 22:52:46.865892 (+ 347us) client_negotiation.cc:240] Negotiation successful
0809 22:52:46.866365 (+ 473us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":523,"thread_start_us":311,"threads_started":1}
I20260809 22:52:46.869089 20959 heartbeater.cc:347] Connected to a master server at 127.15.113.190:34261
I20260809 22:52:46.869400 20959 heartbeater.cc:464] Registering TS with master...
I20260809 22:52:46.870239 20959 heartbeater.cc:511] Master 127.15.113.190:34261 requested a full tablet report, sending...
I20260809 22:52:46.872335 20658 ts_manager.cc:194] Registered new tserver with Master: d55d0555bc8f4f4d980ec1dfe4bfa1ff (127.15.113.129:42369)
WARNING: no policy specified for kudu/127.15.113.130@KRBTEST.COM; defaulting to no policy
I20260809 22:52:46.873808 20658 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.113.129@KRBTEST.COM'} at 127.15.113.129:44523
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.113.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.113.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.130.keytab.
Entry for principal kudu/127.15.113.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.130.keytab.
I20260809 22:52:46.899551 15814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskItYPJR/build/asan/bin/kudu
/tmp/dist-test-taskItYPJR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315935617919-15814-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315935617919-15814-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315935617919-15814-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315935617919-15814-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.113.130:0
--local_ip_for_outbound_sockets=127.15.113.130
--webserver_interface=127.15.113.130
--webserver_port=0
--tserver_master_addrs=127.15.113.190:34261
--keytab_file=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.130.keytab
--principal=kudu/127.15.113.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.113.148:43887
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315935617919-15814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315935617919-15814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315935617919-15814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:52:47.224992 20971 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:52:47.225297 20971 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:52:47.225468 20971 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:52:47.232825 20971 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:52:47.232985 20971 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:52:47.233156 20971 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.113.130
I20260809 22:52:47.242853 20971 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.113.148:43887
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315935617919-15814-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315935617919-15814-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.130.keytab
--principal=kudu/127.15.113.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.113.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315935617919-15814-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.113.130
--webserver_port=0
--tserver_master_addrs=127.15.113.190:34261
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.113.130
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315935617919-15814-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:52:47.244145 20971 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:52:47.245512 20971 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:52:47.258688 20984 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:52:47.258757 20979 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:52:47.258730 20980 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:52:47.259850 20971 server_base.cc:1034] running on GCE node
I20260809 22:52:47.260432 20971 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:52:47.261355 20971 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:52:47.262552 20971 hybrid_clock.cc:648] HybridClock initialized: now 1786315967262504 us; error 46 us; skew 500 ppm
I20260809 22:52:47.262841 20971 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:52:47 dist-test-slave-ncp9 krb5kdc[20585](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315967, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.113.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:52:47.266520 20971 init.cc:377] Logged in from keytab as kudu/127.15.113.130@KRBTEST.COM (short username kudu)
I20260809 22:52:47.267956 20971 webserver.cc:460] Webserver started at http://127.15.113.130:41857/ using document root <none> and password file <none>
I20260809 22:52:47.268586 20971 fs_manager.cc:359] Metadata directory not provided
I20260809 22:52:47.268703 20971 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:52:47.269038 20971 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:52:47.271880 20971 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315935617919-15814-0/minicluster-data/ts-1/data/instance:
uuid: "d64c38a074404117a36bfe9944a52dac"
format_stamp: "Formatted at 2026-08-09 22:52:47 on dist-test-slave-ncp9"
server_key: "43923472fe17dba37893dd12db508d6f"
server_key_iv: "0eb3ab148ab060c402cd25aa834923e4"
server_key_version: "encryptionkey@0"
I20260809 22:52:47.272616 20971 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315935617919-15814-0/minicluster-data/ts-1/wal/instance:
uuid: "d64c38a074404117a36bfe9944a52dac"
format_stamp: "Formatted at 2026-08-09 22:52:47 on dist-test-slave-ncp9"
server_key: "43923472fe17dba37893dd12db508d6f"
server_key_iv: "0eb3ab148ab060c402cd25aa834923e4"
server_key_version: "encryptionkey@0"
I20260809 22:52:47.277670 20971 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260809 22:52:47.280985 20996 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:52:47.282230 20971 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 22:52:47.282379 20971 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315935617919-15814-0/minicluster-data/ts-1/data,/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315935617919-15814-0/minicluster-data/ts-1/wal
uuid: "d64c38a074404117a36bfe9944a52dac"
format_stamp: "Formatted at 2026-08-09 22:52:47 on dist-test-slave-ncp9"
server_key: "43923472fe17dba37893dd12db508d6f"
server_key_iv: "0eb3ab148ab060c402cd25aa834923e4"
server_key_version: "encryptionkey@0"
I20260809 22:52:47.282553 20971 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315935617919-15814-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315935617919-15814-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315935617919-15814-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:52:47.303318 20971 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:52:47.307274 20971 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:52:47.307538 20971 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:52:47.308413 20971 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:52:47.309684 20971 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:52:47.309759 20971 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:52:47.309880 20971 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:52:47.309929 20971 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:52:47.340010 20971 rpc_server.cc:307] RPC server started. Bound to: 127.15.113.130:33225
I20260809 22:52:47.340056 21169 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.113.130:33225 every 8 connection(s)
I20260809 22:52:47.341248 20971 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315935617919-15814-0/minicluster-data/ts-1/data/info.pb
I20260809 22:52:47.346102 15814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskItYPJR/build/asan/bin/kudu as pid 20971
I20260809 22:52:47.346283 15814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315935617919-15814-0/minicluster-data/ts-1/wal/instance
I20260809 22:52:47.346866 15814 external_mini_cluster.cc:1428] Setting key 69b81e58d43df18952b9f738f17aa745
Aug 09 22:52:47 dist-test-slave-ncp9 krb5kdc[20585](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315967, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.113.130@KRBTEST.COM for kudu/127.15.113.190@KRBTEST.COM
I20260809 22:52:47.362324 20963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:52:47.343733 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.113.130:52613 (local address 127.15.113.190:34261)
0809 22:52:47.343942 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0809 22:52:47.343948 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:52:47.345714 (+ 1766us) server_negotiation.cc:407] Connection header received
0809 22:52:47.347942 (+ 2228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:52:47.347947 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:52:47.348011 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:52:47.348092 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:52:47.350043 (+ 1951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:52:47.350759 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:52:47.352749 (+ 1990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:52:47.353234 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:52:47.353369 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:52:47.356562 (+ 3193us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:52:47.356567 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:52:47.356570 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:52:47.356600 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:52:47.359425 (+ 2825us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:52:47.360225 (+ 800us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:52:47.360229 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:52:47.360231 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:52:47.360353 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:52:47.360784 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:52:47.360787 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:52:47.360789 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:52:47.361061 (+ 272us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:52:47.361171 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0809 22:52:47.361903 (+ 732us) server_negotiation.cc:299] Negotiation successful
0809 22:52:47.362072 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260809 22:52:47.363579 21174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:52:47.344212 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:34261 (local address 127.15.113.130:52613)
0809 22:52:47.345558 (+ 1346us) negotiation.cc:107] Waiting for socket to connect
0809 22:52:47.345594 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0809 22:52:47.347700 (+ 2106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:52:47.348355 (+ 655us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:52:47.348365 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:52:47.349101 (+ 736us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:52:47.349861 (+ 760us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:52:47.349886 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:52:47.350914 (+ 1028us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:52:47.350919 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:52:47.352623 (+ 1704us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:52:47.352638 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:52:47.353462 (+ 824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:52:47.353470 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:52:47.353699 (+ 229us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:52:47.354306 (+ 607us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:52:47.354328 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:52:47.356356 (+ 2028us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:52:47.359561 (+ 3205us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:52:47.359569 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:52:47.359584 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:52:47.360131 (+ 547us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:52:47.360452 (+ 321us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:52:47.360456 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:52:47.360459 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:52:47.360691 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:52:47.361182 (+ 491us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:52:47.361191 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:52:47.361555 (+ 364us) client_negotiation.cc:769] Sending connection context
0809 22:52:47.361844 (+ 289us) client_negotiation.cc:240] Negotiation successful
0809 22:52:47.362320 (+ 476us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1112,"thread_start_us":913,"threads_started":1}
I20260809 22:52:47.364970 21170 heartbeater.cc:347] Connected to a master server at 127.15.113.190:34261
I20260809 22:52:47.365267 21170 heartbeater.cc:464] Registering TS with master...
I20260809 22:52:47.366081 21170 heartbeater.cc:511] Master 127.15.113.190:34261 requested a full tablet report, sending...
I20260809 22:52:47.367621 20658 ts_manager.cc:194] Registered new tserver with Master: d64c38a074404117a36bfe9944a52dac (127.15.113.130:33225)
I20260809 22:52:47.368520 20658 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.113.130@KRBTEST.COM'} at 127.15.113.130:52613
WARNING: no policy specified for kudu/127.15.113.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.113.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.113.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.131.keytab.
Entry for principal kudu/127.15.113.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.131.keytab.
I20260809 22:52:47.397547 15814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskItYPJR/build/asan/bin/kudu
/tmp/dist-test-taskItYPJR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315935617919-15814-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315935617919-15814-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315935617919-15814-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315935617919-15814-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.113.131:0
--local_ip_for_outbound_sockets=127.15.113.131
--webserver_interface=127.15.113.131
--webserver_port=0
--tserver_master_addrs=127.15.113.190:34261
--keytab_file=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.131.keytab
--principal=kudu/127.15.113.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.113.148:43887
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315935617919-15814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315935617919-15814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315935617919-15814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:52:47.715732 21177 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:52:47.716014 21177 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:52:47.716157 21177 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:52:47.724400 21177 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:52:47.724517 21177 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:52:47.724637 21177 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.113.131
I20260809 22:52:47.734465 21177 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.113.148:43887
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315935617919-15814-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315935617919-15814-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.131.keytab
--principal=kudu/127.15.113.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.113.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315935617919-15814-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.113.131
--webserver_port=0
--tserver_master_addrs=127.15.113.190:34261
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.113.131
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315935617919-15814-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:52:47.735713 21177 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:52:47.737061 21177 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:52:47.749388 21187 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:52:47.749411 21189 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:52:47.749339 21191 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:52:47.750120 21177 server_base.cc:1034] running on GCE node
I20260809 22:52:47.750665 21177 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:52:47.751479 21177 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:52:47.752628 21177 hybrid_clock.cc:648] HybridClock initialized: now 1786315967752581 us; error 36 us; skew 500 ppm
I20260809 22:52:47.752959 21177 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:52:47 dist-test-slave-ncp9 krb5kdc[20585](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315967, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.113.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:52:47.755960 21177 init.cc:377] Logged in from keytab as kudu/127.15.113.131@KRBTEST.COM (short username kudu)
I20260809 22:52:47.757295 21177 webserver.cc:460] Webserver started at http://127.15.113.131:43797/ using document root <none> and password file <none>
I20260809 22:52:47.757887 21177 fs_manager.cc:359] Metadata directory not provided
I20260809 22:52:47.757995 21177 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:52:47.758257 21177 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:52:47.761039 21177 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315935617919-15814-0/minicluster-data/ts-2/data/instance:
uuid: "a97c6c4004cc46efa36fb9f5dfb101e4"
format_stamp: "Formatted at 2026-08-09 22:52:47 on dist-test-slave-ncp9"
server_key: "908ad67aaf113ef89e116cb6e3f3e3a5"
server_key_iv: "85b8e11dc899cc9ba7356f5d3d6a6a78"
server_key_version: "encryptionkey@0"
I20260809 22:52:47.761772 21177 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315935617919-15814-0/minicluster-data/ts-2/wal/instance:
uuid: "a97c6c4004cc46efa36fb9f5dfb101e4"
format_stamp: "Formatted at 2026-08-09 22:52:47 on dist-test-slave-ncp9"
server_key: "908ad67aaf113ef89e116cb6e3f3e3a5"
server_key_iv: "85b8e11dc899cc9ba7356f5d3d6a6a78"
server_key_version: "encryptionkey@0"
I20260809 22:52:47.766561 21177 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260809 22:52:47.769740 21198 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:52:47.770920 21177 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.001s
I20260809 22:52:47.771051 21177 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315935617919-15814-0/minicluster-data/ts-2/data,/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315935617919-15814-0/minicluster-data/ts-2/wal
uuid: "a97c6c4004cc46efa36fb9f5dfb101e4"
format_stamp: "Formatted at 2026-08-09 22:52:47 on dist-test-slave-ncp9"
server_key: "908ad67aaf113ef89e116cb6e3f3e3a5"
server_key_iv: "85b8e11dc899cc9ba7356f5d3d6a6a78"
server_key_version: "encryptionkey@0"
I20260809 22:52:47.771200 21177 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315935617919-15814-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315935617919-15814-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315935617919-15814-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:52:47.791289 21177 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:52:47.795219 21177 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:52:47.795467 21177 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:52:47.796319 21177 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:52:47.797668 21177 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:52:47.797751 21177 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:52:47.797868 21177 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:52:47.797920 21177 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:52:47.835444 21177 rpc_server.cc:307] RPC server started. Bound to: 127.15.113.131:36223
I20260809 22:52:47.835464 21382 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.113.131:36223 every 8 connection(s)
I20260809 22:52:47.836753 21177 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315935617919-15814-0/minicluster-data/ts-2/data/info.pb
I20260809 22:52:47.846218 15814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskItYPJR/build/asan/bin/kudu as pid 21177
I20260809 22:52:47.846418 15814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315935617919-15814-0/minicluster-data/ts-2/wal/instance
I20260809 22:52:47.847040 15814 external_mini_cluster.cc:1428] Setting key baa0fc50853b14d2b43b469cc9d9c98f
Aug 09 22:52:47 dist-test-slave-ncp9 krb5kdc[20585](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315967, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.113.131@KRBTEST.COM for kudu/127.15.113.190@KRBTEST.COM
I20260809 22:52:47.857326 20963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:52:47.840255 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.113.131:45431 (local address 127.15.113.190:34261)
0809 22:52:47.840434 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0809 22:52:47.840439 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:52:47.841771 (+ 1332us) server_negotiation.cc:407] Connection header received
0809 22:52:47.843510 (+ 1739us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:52:47.843514 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:52:47.843582 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:52:47.843660 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:52:47.845072 (+ 1412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:52:47.846108 (+ 1036us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:52:47.848481 (+ 2373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:52:47.849096 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:52:47.849218 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:52:47.852475 (+ 3257us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:52:47.852480 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:52:47.852483 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:52:47.852515 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:52:47.854900 (+ 2385us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:52:47.855511 (+ 611us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:52:47.855515 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:52:47.855517 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:52:47.855632 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:52:47.855945 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:52:47.855948 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:52:47.855950 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:52:47.856203 (+ 253us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:52:47.856312 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0809 22:52:47.856880 (+ 568us) server_negotiation.cc:299] Negotiation successful
0809 22:52:47.857076 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260809 22:52:47.858181 21387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:52:47.840885 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:34261 (local address 127.15.113.131:45431)
0809 22:52:47.841609 (+ 724us) negotiation.cc:107] Waiting for socket to connect
0809 22:52:47.841657 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0809 22:52:47.843238 (+ 1581us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:52:47.843849 (+ 611us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:52:47.843856 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:52:47.844361 (+ 505us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:52:47.844924 (+ 563us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:52:47.844948 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:52:47.846325 (+ 1377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:52:47.846331 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:52:47.848319 (+ 1988us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:52:47.848334 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:52:47.849329 (+ 995us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:52:47.849337 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:52:47.849548 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:52:47.850174 (+ 626us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:52:47.850196 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:52:47.852298 (+ 2102us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:52:47.855019 (+ 2721us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:52:47.855026 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:52:47.855037 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:52:47.855415 (+ 378us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:52:47.855705 (+ 290us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:52:47.855708 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:52:47.855710 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:52:47.855872 (+ 162us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:52:47.856306 (+ 434us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:52:47.856312 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:52:47.856612 (+ 300us) client_negotiation.cc:769] Sending connection context
0809 22:52:47.856822 (+ 210us) client_negotiation.cc:240] Negotiation successful
0809 22:52:47.857197 (+ 375us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":468,"thread_start_us":256,"threads_started":1}
I20260809 22:52:47.859375 21383 heartbeater.cc:347] Connected to a master server at 127.15.113.190:34261
I20260809 22:52:47.859614 21383 heartbeater.cc:464] Registering TS with master...
I20260809 22:52:47.860250 21383 heartbeater.cc:511] Master 127.15.113.190:34261 requested a full tablet report, sending...
I20260809 22:52:47.861685 20658 ts_manager.cc:194] Registered new tserver with Master: a97c6c4004cc46efa36fb9f5dfb101e4 (127.15.113.131:36223)
I20260809 22:52:47.862519 15814 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 22:52:47.862738 20658 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.113.131@KRBTEST.COM'} at 127.15.113.131:45431
I20260809 22:52:47.876518 20959 heartbeater.cc:503] Master 127.15.113.190:34261 was elected leader, sending a full tablet report...
Aug 09 22:52:47 dist-test-slave-ncp9 krb5kdc[20585](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315967, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Aug 09 22:52:47 dist-test-slave-ncp9 krb5kdc[20585](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315967, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.15.113.190@KRBTEST.COM
I20260809 22:52:47.916630 20963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:52:47.902514 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52656 (local address 127.15.113.190:34261)
0809 22:52:47.902741 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0809 22:52:47.902748 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:52:47.903142 (+ 394us) server_negotiation.cc:407] Connection header received
0809 22:52:47.903353 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:52:47.903357 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:52:47.903427 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:52:47.903531 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:52:47.905057 (+ 1526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:52:47.905833 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:52:47.907165 (+ 1332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:52:47.907675 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:52:47.907919 (+ 244us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:52:47.911681 (+ 3762us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:52:47.911685 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:52:47.911689 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:52:47.911721 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:52:47.914207 (+ 2486us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:52:47.914816 (+ 609us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:52:47.914820 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:52:47.914822 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:52:47.914949 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:52:47.915301 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:52:47.915304 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:52:47.915306 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:52:47.915631 (+ 325us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:52:47.915855 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0809 22:52:47.916125 (+ 270us) server_negotiation.cc:299] Negotiation successful
0809 22:52:47.916330 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
W20260809 22:52:47.917213 20658 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:52656
I20260809 22:52:47.919094 15814 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskItYPJR/build/asan/bin/kudu pid 20755
Aug 09 22:52:47 dist-test-slave-ncp9 krb5kdc[20585](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315967, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.15.113.129@KRBTEST.COM
I20260809 22:52:47.933470 21407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:52:47.919991 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44410 (local address 127.15.113.129:42369)
0809 22:52:47.920581 (+ 590us) server_negotiation.cc:206] Beginning negotiation
0809 22:52:47.920588 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:52:47.920612 (+ 24us) server_negotiation.cc:407] Connection header received
0809 22:52:47.920675 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:52:47.920682 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:52:47.920879 (+ 197us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:52:47.921054 (+ 175us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:52:47.922097 (+ 1043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:52:47.923103 (+ 1006us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:52:47.924210 (+ 1107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:52:47.924763 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:52:47.924856 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:52:47.928284 (+ 3428us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:52:47.928297 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:52:47.928311 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:52:47.928346 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:52:47.930873 (+ 2527us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:52:47.931487 (+ 614us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:52:47.931493 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:52:47.931495 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:52:47.931617 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:52:47.931965 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:52:47.931968 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:52:47.931970 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:52:47.932299 (+ 329us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:52:47.932567 (+ 268us) server_negotiation.cc:1035] Waiting for connection context
0809 22:52:47.932870 (+ 303us) server_negotiation.cc:299] Negotiation successful
0809 22:52:47.933160 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":444,"thread_start_us":285,"threads_started":1}
W20260809 22:52:47.934643 20954 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:44410
W20260809 22:52:47.935680 15814 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskItYPJR/build/asan/bin/kudu pid 20755: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 22:52:47.935868 15814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskItYPJR/build/asan/bin/kudu with pid 20755
I20260809 22:52:47.959992 15814 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskItYPJR/build/asan/bin/kudu pid 20971
Aug 09 22:52:47 dist-test-slave-ncp9 krb5kdc[20585](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315967, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.15.113.130@KRBTEST.COM
I20260809 22:52:47.975348 21408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:52:47.960774 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52492 (local address 127.15.113.130:33225)
0809 22:52:47.961355 (+ 581us) server_negotiation.cc:206] Beginning negotiation
0809 22:52:47.961363 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 22:52:47.961388 (+ 25us) server_negotiation.cc:407] Connection header received
0809 22:52:47.961449 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:52:47.961456 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:52:47.961663 (+ 207us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:52:47.961808 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:52:47.963107 (+ 1299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:52:47.964116 (+ 1009us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:52:47.965450 (+ 1334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:52:47.965988 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:52:47.966103 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:52:47.970026 (+ 3923us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:52:47.970037 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:52:47.970051 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:52:47.970092 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:52:47.972546 (+ 2454us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:52:47.973219 (+ 673us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:52:47.973226 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:52:47.973228 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:52:47.973358 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:52:47.973734 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:52:47.973737 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:52:47.973739 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:52:47.974107 (+ 368us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:52:47.974419 (+ 312us) server_negotiation.cc:1035] Waiting for connection context
0809 22:52:47.974753 (+ 334us) server_negotiation.cc:299] Negotiation successful
0809 22:52:47.975023 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":433,"thread_start_us":299,"threads_started":1}
W20260809 22:52:47.976598 21163 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:52492
W20260809 22:52:47.977721 15814 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskItYPJR/build/asan/bin/kudu pid 20971: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 22:52:47.977862 15814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskItYPJR/build/asan/bin/kudu with pid 20971
I20260809 22:52:47.997944 15814 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskItYPJR/build/asan/bin/kudu pid 21177
Aug 09 22:52:48 dist-test-slave-ncp9 krb5kdc[20585](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315967, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.15.113.131@KRBTEST.COM
I20260809 22:52:48.013535 21412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:52:47.998775 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56674 (local address 127.15.113.131:36223)
0809 22:52:47.999330 (+ 555us) server_negotiation.cc:206] Beginning negotiation
0809 22:52:47.999336 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:52:47.999365 (+ 29us) server_negotiation.cc:407] Connection header received
0809 22:52:47.999427 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:52:47.999433 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:52:47.999629 (+ 196us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:52:47.999781 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:52:48.001113 (+ 1332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:52:48.002156 (+ 1043us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:52:48.003489 (+ 1333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:52:48.004026 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:52:48.004136 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:52:48.008285 (+ 4149us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:52:48.008297 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:52:48.008311 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:52:48.008342 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:52:48.010852 (+ 2510us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:52:48.011466 (+ 614us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:52:48.011474 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:52:48.011476 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:52:48.011596 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:52:48.011933 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:52:48.011937 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:52:48.011939 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:52:48.012290 (+ 351us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:52:48.012581 (+ 291us) server_negotiation.cc:1035] Waiting for connection context
0809 22:52:48.012872 (+ 291us) server_negotiation.cc:299] Negotiation successful
0809 22:52:48.013257 (+ 385us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":400,"thread_start_us":241,"threads_started":1}
W20260809 22:52:48.014757 21378 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:56674
W20260809 22:52:48.015808 15814 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskItYPJR/build/asan/bin/kudu pid 21177: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 22:52:48.015969 15814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskItYPJR/build/asan/bin/kudu with pid 21177
I20260809 22:52:48.041224 15814 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskItYPJR/build/asan/bin/kudu pid 20613
I20260809 22:52:48.042390 20710 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:52:48.144833 15814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskItYPJR/build/asan/bin/kudu with pid 20613
2026-08-09T22:52:48Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (3659 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315935617919-15814-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 22:52:48 dist-test-slave-ncp9 krb5kdc[21421](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 22:52:48 dist-test-slave-ncp9 krb5kdc[21421](info): set up 2 sockets
Aug 09 22:52:48 dist-test-slave-ncp9 krb5kdc[21421](info): commencing operation
krb5kdc: starting...
W20260809 22:52:49.334900 15814 mini_kdc.cc:121] Time spent starting KDC: real 1.142s user 0.004s sys 0.020s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315935617919-15814-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315935617919-15814-0/krb5kdc/test-user.keytab.
Aug 09 22:52:49 dist-test-slave-ncp9 krb5kdc[21421](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315969, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T22:52:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T22:52:49Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.113.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.113.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.113.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.190.keytab.
Entry for principal kudu/127.15.113.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.190.keytab.
I20260809 22:52:49.542649 15814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskItYPJR/build/asan/bin/kudu
/tmp/dist-test-taskItYPJR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315935617919-15814-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315935617919-15814-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315935617919-15814-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315935617919-15814-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.113.190:37295
--webserver_interface=127.15.113.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.190.keytab
--principal=kudu/127.15.113.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.113.148:43167
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.113.190:37295
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315935617919-15814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315935617919-15814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315935617919-15814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:52:49.862548 21451 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:52:49.862840 21451 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:52:49.862980 21451 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:52:49.869915 21451 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 22:52:49.870020 21451 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:52:49.870097 21451 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:52:49.870182 21451 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 22:52:49.870245 21451 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 22:52:49.879608 21451 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.113.148:43167
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315935617919-15814-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315935617919-15814-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.113.190:37295
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.190.keytab
--principal=kudu/127.15.113.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.113.190:37295
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315935617919-15814-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.113.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315935617919-15814-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:52:49.880824 21451 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:52:49.882092 21451 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:52:49.893154 21462 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:52:49.893194 21466 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:52:49.893235 21463 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:52:49.894330 21451 server_base.cc:1034] running on GCE node
I20260809 22:52:49.895033 21451 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:52:49.896343 21451 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:52:49.897540 21451 hybrid_clock.cc:648] HybridClock initialized: now 1786315969897493 us; error 34 us; skew 500 ppm
I20260809 22:52:49.897869 21451 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:52:49 dist-test-slave-ncp9 krb5kdc[21421](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315969, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.113.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:52:49.905059 21451 init.cc:377] Logged in from keytab as kudu/127.15.113.190@KRBTEST.COM (short username kudu)
I20260809 22:52:49.906409 21451 webserver.cc:460] Webserver started at http://127.15.113.190:34253/ using document root <none> and password file <none>
I20260809 22:52:49.907084 21451 fs_manager.cc:359] Metadata directory not provided
I20260809 22:52:49.907202 21451 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:52:49.907491 21451 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:52:49.910462 21451 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315935617919-15814-0/minicluster-data/master-0/data/instance:
uuid: "ac3340c0fdae412a8f28f17df96ca5ed"
format_stamp: "Formatted at 2026-08-09 22:52:49 on dist-test-slave-ncp9"
server_key: "350fbfbc7daf5d422c8cd63a8ca292bd"
server_key_iv: "044101389cc520d36162e7268ce6e48c"
server_key_version: "encryptionkey@0"
I20260809 22:52:49.911258 21451 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315935617919-15814-0/minicluster-data/master-0/wal/instance:
uuid: "ac3340c0fdae412a8f28f17df96ca5ed"
format_stamp: "Formatted at 2026-08-09 22:52:49 on dist-test-slave-ncp9"
server_key: "350fbfbc7daf5d422c8cd63a8ca292bd"
server_key_iv: "044101389cc520d36162e7268ce6e48c"
server_key_version: "encryptionkey@0"
I20260809 22:52:49.916790 21451 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260809 22:52:49.920068 21481 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:52:49.921387 21451 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 22:52:49.921551 21451 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315935617919-15814-0/minicluster-data/master-0/data,/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315935617919-15814-0/minicluster-data/master-0/wal
uuid: "ac3340c0fdae412a8f28f17df96ca5ed"
format_stamp: "Formatted at 2026-08-09 22:52:49 on dist-test-slave-ncp9"
server_key: "350fbfbc7daf5d422c8cd63a8ca292bd"
server_key_iv: "044101389cc520d36162e7268ce6e48c"
server_key_version: "encryptionkey@0"
I20260809 22:52:49.921733 21451 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315935617919-15814-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315935617919-15814-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315935617919-15814-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:52:49.939471 21451 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:52:49.943537 21451 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:52:49.943794 21451 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:52:49.960745 21451 rpc_server.cc:307] RPC server started. Bound to: 127.15.113.190:37295
I20260809 22:52:49.960764 21554 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.113.190:37295 every 8 connection(s)
I20260809 22:52:49.962023 21451 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315935617919-15814-0/minicluster-data/master-0/data/info.pb
I20260809 22:52:49.965626 21556 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:52:49.969975 15814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskItYPJR/build/asan/bin/kudu as pid 21451
I20260809 22:52:49.970227 15814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315935617919-15814-0/minicluster-data/master-0/wal/instance
I20260809 22:52:49.971033 15814 external_mini_cluster.cc:1428] Setting key 1f2595965785776806a6fc10a688b897
I20260809 22:52:49.976791 21556 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ac3340c0fdae412a8f28f17df96ca5ed: Bootstrap starting.
I20260809 22:52:49.980291 21556 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ac3340c0fdae412a8f28f17df96ca5ed: Neither blocks nor log segments found. Creating new log.
I20260809 22:52:49.981518 21556 log.cc:824] T 00000000000000000000000000000000 P ac3340c0fdae412a8f28f17df96ca5ed: Log is configured to *not* fsync() on all Append() calls
I20260809 22:52:49.984819 21556 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ac3340c0fdae412a8f28f17df96ca5ed: No bootstrap required, opened a new log
Aug 09 22:52:49 dist-test-slave-ncp9 krb5kdc[21421](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315969, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.113.190@KRBTEST.COM
I20260809 22:52:49.992461 21556 raft_consensus.cc:348] T 00000000000000000000000000000000 P ac3340c0fdae412a8f28f17df96ca5ed [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac3340c0fdae412a8f28f17df96ca5ed" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 37295 } }
I20260809 22:52:49.992911 21556 raft_consensus.cc:374] T 00000000000000000000000000000000 P ac3340c0fdae412a8f28f17df96ca5ed [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:52:49.993014 21556 raft_consensus.cc:729] T 00000000000000000000000000000000 P ac3340c0fdae412a8f28f17df96ca5ed [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ac3340c0fdae412a8f28f17df96ca5ed, State: Initialized, Role: FOLLOWER
I20260809 22:52:49.993557 21556 consensus_queue.cc:260] T 00000000000000000000000000000000 P ac3340c0fdae412a8f28f17df96ca5ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac3340c0fdae412a8f28f17df96ca5ed" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 37295 } }
I20260809 22:52:49.993884 21556 raft_consensus.cc:388] T 00000000000000000000000000000000 P ac3340c0fdae412a8f28f17df96ca5ed [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 22:52:49.993906 21561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:52:49.973025 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33382 (local address 127.15.113.190:37295)
0809 22:52:49.973812 (+ 787us) server_negotiation.cc:206] Beginning negotiation
0809 22:52:49.973823 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 22:52:49.973867 (+ 44us) server_negotiation.cc:407] Connection header received
0809 22:52:49.975611 (+ 1744us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:52:49.975643 (+ 32us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:52:49.976004 (+ 361us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:52:49.976316 (+ 312us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:52:49.978138 (+ 1822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:52:49.979445 (+ 1307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:52:49.981221 (+ 1776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:52:49.981973 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:52:49.982195 (+ 222us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:52:49.986645 (+ 4450us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:52:49.986653 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:52:49.986667 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:52:49.986712 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:52:49.989734 (+ 3022us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:52:49.990418 (+ 684us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:52:49.990425 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:52:49.990431 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:52:49.990601 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:52:49.990991 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:52:49.990995 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:52:49.990997 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:52:49.991546 (+ 549us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:52:49.991882 (+ 336us) server_negotiation.cc:1035] Waiting for connection context
0809 22:52:49.992286 (+ 404us) server_negotiation.cc:299] Negotiation successful
0809 22:52:49.992698 (+ 412us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":565,"thread_start_us":344,"threads_started":1}
I20260809 22:52:49.994040 21556 raft_consensus.cc:482] T 00000000000000000000000000000000 P ac3340c0fdae412a8f28f17df96ca5ed [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 22:52:49.994257 21556 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ac3340c0fdae412a8f28f17df96ca5ed [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:52:49.996505 21556 raft_consensus.cc:504] T 00000000000000000000000000000000 P ac3340c0fdae412a8f28f17df96ca5ed [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac3340c0fdae412a8f28f17df96ca5ed" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 37295 } }
I20260809 22:52:49.997112 21556 leader_election.cc:302] T 00000000000000000000000000000000 P ac3340c0fdae412a8f28f17df96ca5ed [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ac3340c0fdae412a8f28f17df96ca5ed; no voters:
I20260809 22:52:49.997603 21556 leader_election.cc:288] T 00000000000000000000000000000000 P ac3340c0fdae412a8f28f17df96ca5ed [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 22:52:49.997726 21563 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ac3340c0fdae412a8f28f17df96ca5ed [term 1 FOLLOWER]: Leader election won for term 1
I20260809 22:52:49.998373 21563 raft_consensus.cc:686] T 00000000000000000000000000000000 P ac3340c0fdae412a8f28f17df96ca5ed [term 1 LEADER]: Becoming Leader. State: Replica: ac3340c0fdae412a8f28f17df96ca5ed, State: Running, Role: LEADER
I20260809 22:52:49.998919 21563 consensus_queue.cc:237] T 00000000000000000000000000000000 P ac3340c0fdae412a8f28f17df96ca5ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac3340c0fdae412a8f28f17df96ca5ed" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 37295 } }
I20260809 22:52:49.999526 21556 sys_catalog.cc:564] T 00000000000000000000000000000000 P ac3340c0fdae412a8f28f17df96ca5ed [sys.catalog]: configured and running, proceeding with master startup.
I20260809 22:52:50.001843 21564 sys_catalog.cc:455] T 00000000000000000000000000000000 P ac3340c0fdae412a8f28f17df96ca5ed [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ac3340c0fdae412a8f28f17df96ca5ed" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac3340c0fdae412a8f28f17df96ca5ed" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 37295 } } }
I20260809 22:52:50.001907 21565 sys_catalog.cc:455] T 00000000000000000000000000000000 P ac3340c0fdae412a8f28f17df96ca5ed [sys.catalog]: SysCatalogTable state changed. Reason: New leader ac3340c0fdae412a8f28f17df96ca5ed. Latest consensus state: current_term: 1 leader_uuid: "ac3340c0fdae412a8f28f17df96ca5ed" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac3340c0fdae412a8f28f17df96ca5ed" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 37295 } } }
I20260809 22:52:50.002260 21565 sys_catalog.cc:458] T 00000000000000000000000000000000 P ac3340c0fdae412a8f28f17df96ca5ed [sys.catalog]: This master's current role is: LEADER
I20260809 22:52:50.002360 21564 sys_catalog.cc:458] T 00000000000000000000000000000000 P ac3340c0fdae412a8f28f17df96ca5ed [sys.catalog]: This master's current role is: LEADER
I20260809 22:52:50.003103 21573 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 22:52:50.008209 21573 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 22:52:50.017409 21573 catalog_manager.cc:1347] Generated new cluster ID: 803a64fdeed641d1a2b6cf8e041cdad8
I20260809 22:52:50.017522 21573 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 22:52:50.038393 21573 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 22:52:50.040215 21573 catalog_manager.cc:1504] Loading token signing keys...
I20260809 22:52:50.063470 21573 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ac3340c0fdae412a8f28f17df96ca5ed: Generated new TSK 0
I20260809 22:52:50.064530 21573 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.113.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.113.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.113.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.129.keytab.
Entry for principal kudu/127.15.113.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.129.keytab.
I20260809 22:52:50.125803 15814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskItYPJR/build/asan/bin/kudu
/tmp/dist-test-taskItYPJR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315935617919-15814-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315935617919-15814-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315935617919-15814-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315935617919-15814-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.113.129:0
--local_ip_for_outbound_sockets=127.15.113.129
--webserver_interface=127.15.113.129
--webserver_port=0
--tserver_master_addrs=127.15.113.190:37295
--keytab_file=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.129.keytab
--principal=kudu/127.15.113.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.113.148:43167
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315935617919-15814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315935617919-15814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315935617919-15814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:52:50.450192 21593 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:52:50.450451 21593 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:52:50.450551 21593 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:52:50.457549 21593 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:52:50.457648 21593 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:52:50.457762 21593 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.113.129
I20260809 22:52:50.466991 21593 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.113.148:43167
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315935617919-15814-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315935617919-15814-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.129.keytab
--principal=kudu/127.15.113.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.113.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315935617919-15814-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.113.129
--webserver_port=0
--tserver_master_addrs=127.15.113.190:37295
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.113.129
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315935617919-15814-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:52:50.468135 21593 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:52:50.469390 21593 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:52:50.481819 21598 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:52:50.481913 21604 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:52:50.481876 21600 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:52:50.482666 21593 server_base.cc:1034] running on GCE node
I20260809 22:52:50.483228 21593 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:52:50.484126 21593 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:52:50.485318 21593 hybrid_clock.cc:648] HybridClock initialized: now 1786315970485272 us; error 37 us; skew 500 ppm
I20260809 22:52:50.485591 21593 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:52:50 dist-test-slave-ncp9 krb5kdc[21421](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315970, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.113.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:52:50.488494 21593 init.cc:377] Logged in from keytab as kudu/127.15.113.129@KRBTEST.COM (short username kudu)
I20260809 22:52:50.489924 21593 webserver.cc:460] Webserver started at http://127.15.113.129:41121/ using document root <none> and password file <none>
I20260809 22:52:50.490566 21593 fs_manager.cc:359] Metadata directory not provided
I20260809 22:52:50.490681 21593 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:52:50.490973 21593 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:52:50.493799 21593 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315935617919-15814-0/minicluster-data/ts-0/data/instance:
uuid: "fe4be91562d7452587835b77cda2b1b5"
format_stamp: "Formatted at 2026-08-09 22:52:50 on dist-test-slave-ncp9"
server_key: "bd99793391180f56749031e6fbcba855"
server_key_iv: "2eb4fd66b41b001601ec4891a1cd50f5"
server_key_version: "encryptionkey@0"
I20260809 22:52:50.494601 21593 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315935617919-15814-0/minicluster-data/ts-0/wal/instance:
uuid: "fe4be91562d7452587835b77cda2b1b5"
format_stamp: "Formatted at 2026-08-09 22:52:50 on dist-test-slave-ncp9"
server_key: "bd99793391180f56749031e6fbcba855"
server_key_iv: "2eb4fd66b41b001601ec4891a1cd50f5"
server_key_version: "encryptionkey@0"
I20260809 22:52:50.499495 21593 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 22:52:50.502955 21613 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:52:50.504223 21593 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 22:52:50.504370 21593 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315935617919-15814-0/minicluster-data/ts-0/data,/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315935617919-15814-0/minicluster-data/ts-0/wal
uuid: "fe4be91562d7452587835b77cda2b1b5"
format_stamp: "Formatted at 2026-08-09 22:52:50 on dist-test-slave-ncp9"
server_key: "bd99793391180f56749031e6fbcba855"
server_key_iv: "2eb4fd66b41b001601ec4891a1cd50f5"
server_key_version: "encryptionkey@0"
I20260809 22:52:50.504554 21593 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315935617919-15814-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315935617919-15814-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315935617919-15814-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:52:50.519317 21593 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:52:50.523051 21593 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:52:50.523319 21593 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:52:50.524233 21593 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:52:50.525632 21593 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:52:50.525712 21593 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:52:50.525828 21593 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:52:50.525883 21593 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:52:50.554979 21593 rpc_server.cc:307] RPC server started. Bound to: 127.15.113.129:35591
I20260809 22:52:50.555042 21806 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.113.129:35591 every 8 connection(s)
I20260809 22:52:50.556213 21593 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315935617919-15814-0/minicluster-data/ts-0/data/info.pb
I20260809 22:52:50.565300 15814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskItYPJR/build/asan/bin/kudu as pid 21593
I20260809 22:52:50.565487 15814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315935617919-15814-0/minicluster-data/ts-0/wal/instance
I20260809 22:52:50.566134 15814 external_mini_cluster.cc:1428] Setting key 97b35319bb32257c5eba1bccd1e1827f
Aug 09 22:52:50 dist-test-slave-ncp9 krb5kdc[21421](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315970, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.113.129@KRBTEST.COM for kudu/127.15.113.190@KRBTEST.COM
I20260809 22:52:50.576135 21812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:52:50.558624 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.113.129:55663 (local address 127.15.113.190:37295)
0809 22:52:50.559050 (+ 426us) server_negotiation.cc:206] Beginning negotiation
0809 22:52:50.559055 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:52:50.559984 (+ 929us) server_negotiation.cc:407] Connection header received
0809 22:52:50.562045 (+ 2061us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:52:50.562052 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:52:50.562153 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:52:50.562282 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:52:50.563774 (+ 1492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:52:50.564490 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:52:50.566077 (+ 1587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:52:50.566696 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:52:50.566817 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:52:50.570442 (+ 3625us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:52:50.570449 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:52:50.570454 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:52:50.570494 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:52:50.573412 (+ 2918us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:52:50.574041 (+ 629us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:52:50.574046 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:52:50.574050 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:52:50.574196 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:52:50.574567 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:52:50.574571 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:52:50.574574 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:52:50.574888 (+ 314us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:52:50.575033 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0809 22:52:50.575605 (+ 572us) server_negotiation.cc:299] Negotiation successful
0809 22:52:50.575821 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":318,"thread_start_us":218,"threads_started":1}
I20260809 22:52:50.576838 21815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:52:50.559142 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:37295 (local address 127.15.113.129:55663)
0809 22:52:50.559824 (+ 682us) negotiation.cc:107] Waiting for socket to connect
0809 22:52:50.559866 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0809 22:52:50.561728 (+ 1862us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:52:50.562517 (+ 789us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:52:50.562524 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:52:50.563014 (+ 490us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:52:50.563638 (+ 624us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:52:50.563656 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:52:50.564607 (+ 951us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:52:50.564610 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:52:50.565916 (+ 1306us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:52:50.565928 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:52:50.566926 (+ 998us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:52:50.566935 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:52:50.567171 (+ 236us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:52:50.567990 (+ 819us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:52:50.568019 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:52:50.570230 (+ 2211us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:52:50.573530 (+ 3300us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:52:50.573536 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:52:50.573547 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:52:50.573922 (+ 375us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:52:50.574289 (+ 367us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:52:50.574292 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:52:50.574294 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:52:50.574467 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:52:50.574985 (+ 518us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:52:50.574990 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:52:50.575270 (+ 280us) client_negotiation.cc:769] Sending connection context
0809 22:52:50.575467 (+ 197us) client_negotiation.cc:240] Negotiation successful
0809 22:52:50.575805 (+ 338us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":444,"thread_start_us":249,"threads_started":1}
I20260809 22:52:50.578321 21809 heartbeater.cc:347] Connected to a master server at 127.15.113.190:37295
I20260809 22:52:50.578595 21809 heartbeater.cc:464] Registering TS with master...
I20260809 22:52:50.579276 21809 heartbeater.cc:511] Master 127.15.113.190:37295 requested a full tablet report, sending...
I20260809 22:52:50.581272 21503 ts_manager.cc:194] Registered new tserver with Master: fe4be91562d7452587835b77cda2b1b5 (127.15.113.129:35591)
I20260809 22:52:50.582647 21503 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.113.129@KRBTEST.COM'} at 127.15.113.129:55663
WARNING: no policy specified for kudu/127.15.113.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.113.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.113.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.130.keytab.
Entry for principal kudu/127.15.113.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.130.keytab.
I20260809 22:52:50.613911 15814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskItYPJR/build/asan/bin/kudu
/tmp/dist-test-taskItYPJR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315935617919-15814-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315935617919-15814-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315935617919-15814-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315935617919-15814-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.113.130:0
--local_ip_for_outbound_sockets=127.15.113.130
--webserver_interface=127.15.113.130
--webserver_port=0
--tserver_master_addrs=127.15.113.190:37295
--keytab_file=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.130.keytab
--principal=kudu/127.15.113.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.113.148:43167
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315935617919-15814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315935617919-15814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315935617919-15814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:52:50.936404 21818 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:52:50.936714 21818 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:52:50.936870 21818 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:52:50.944298 21818 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:52:50.944427 21818 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:52:50.944617 21818 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.113.130
I20260809 22:52:50.954490 21818 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.113.148:43167
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315935617919-15814-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315935617919-15814-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.130.keytab
--principal=kudu/127.15.113.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.113.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315935617919-15814-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.113.130
--webserver_port=0
--tserver_master_addrs=127.15.113.190:37295
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.113.130
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315935617919-15814-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:52:50.955772 21818 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:52:50.957217 21818 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:52:50.972607 21824 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:52:50.973353 21825 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:52:50.974278 21830 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:52:50.975126 21818 server_base.cc:1034] running on GCE node
I20260809 22:52:50.975754 21818 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:52:50.976728 21818 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:52:50.977949 21818 hybrid_clock.cc:648] HybridClock initialized: now 1786315970977874 us; error 52 us; skew 500 ppm
I20260809 22:52:50.978251 21818 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:52:50 dist-test-slave-ncp9 krb5kdc[21421](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315970, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.113.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:52:50.981438 21818 init.cc:377] Logged in from keytab as kudu/127.15.113.130@KRBTEST.COM (short username kudu)
I20260809 22:52:50.982981 21818 webserver.cc:460] Webserver started at http://127.15.113.130:45509/ using document root <none> and password file <none>
I20260809 22:52:50.983711 21818 fs_manager.cc:359] Metadata directory not provided
I20260809 22:52:50.983830 21818 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:52:50.984122 21818 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:52:50.987043 21818 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315935617919-15814-0/minicluster-data/ts-1/data/instance:
uuid: "b83de4e843eb425896ab5fc2cd0b41f6"
format_stamp: "Formatted at 2026-08-09 22:52:50 on dist-test-slave-ncp9"
server_key: "c61619a20f8eaa27e48fd8d35c76d2c8"
server_key_iv: "841aeefbd8975e50ea8a30a0ac14518b"
server_key_version: "encryptionkey@0"
I20260809 22:52:50.987831 21818 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315935617919-15814-0/minicluster-data/ts-1/wal/instance:
uuid: "b83de4e843eb425896ab5fc2cd0b41f6"
format_stamp: "Formatted at 2026-08-09 22:52:50 on dist-test-slave-ncp9"
server_key: "c61619a20f8eaa27e48fd8d35c76d2c8"
server_key_iv: "841aeefbd8975e50ea8a30a0ac14518b"
server_key_version: "encryptionkey@0"
I20260809 22:52:50.992791 21818 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260809 22:52:50.996161 21852 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:52:50.997452 21818 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 22:52:50.997615 21818 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315935617919-15814-0/minicluster-data/ts-1/data,/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315935617919-15814-0/minicluster-data/ts-1/wal
uuid: "b83de4e843eb425896ab5fc2cd0b41f6"
format_stamp: "Formatted at 2026-08-09 22:52:50 on dist-test-slave-ncp9"
server_key: "c61619a20f8eaa27e48fd8d35c76d2c8"
server_key_iv: "841aeefbd8975e50ea8a30a0ac14518b"
server_key_version: "encryptionkey@0"
I20260809 22:52:50.997793 21818 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315935617919-15814-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315935617919-15814-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315935617919-15814-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:52:51.013583 21818 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:52:51.017271 21818 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:52:51.017525 21818 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:52:51.018414 21818 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:52:51.019695 21818 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:52:51.019770 21818 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:52:51.019888 21818 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:52:51.019944 21818 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:52:51.050339 21818 rpc_server.cc:307] RPC server started. Bound to: 127.15.113.130:33259
I20260809 22:52:51.050357 22046 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.113.130:33259 every 8 connection(s)
I20260809 22:52:51.051595 21818 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315935617919-15814-0/minicluster-data/ts-1/data/info.pb
I20260809 22:52:51.052354 15814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskItYPJR/build/asan/bin/kudu as pid 21818
I20260809 22:52:51.052546 15814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315935617919-15814-0/minicluster-data/ts-1/wal/instance
I20260809 22:52:51.053115 15814 external_mini_cluster.cc:1428] Setting key ec3c338825a4800dcea5f2f9765cf8e2
Aug 09 22:52:51 dist-test-slave-ncp9 krb5kdc[21421](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315970, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.113.130@KRBTEST.COM for kudu/127.15.113.190@KRBTEST.COM
I20260809 22:52:51.073271 21812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:52:51.054377 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.113.130:44709 (local address 127.15.113.190:37295)
0809 22:52:51.054535 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0809 22:52:51.054540 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:52:51.055999 (+ 1459us) server_negotiation.cc:407] Connection header received
0809 22:52:51.058451 (+ 2452us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:52:51.058456 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:52:51.058534 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:52:51.058618 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:52:51.060475 (+ 1857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:52:51.061252 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:52:51.063283 (+ 2031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:52:51.063769 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:52:51.063925 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:52:51.067783 (+ 3858us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:52:51.067788 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:52:51.067792 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:52:51.067822 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:52:51.070257 (+ 2435us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:52:51.071117 (+ 860us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:52:51.071120 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:52:51.071123 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:52:51.071235 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:52:51.071671 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:52:51.071674 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:52:51.071676 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:52:51.071907 (+ 231us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:52:51.072010 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0809 22:52:51.072826 (+ 816us) server_negotiation.cc:299] Negotiation successful
0809 22:52:51.073026 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260809 22:52:51.074601 22049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:52:51.054937 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:37295 (local address 127.15.113.130:44709)
0809 22:52:51.055799 (+ 862us) negotiation.cc:107] Waiting for socket to connect
0809 22:52:51.055862 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0809 22:52:51.058169 (+ 2307us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:52:51.058878 (+ 709us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:52:51.058889 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:52:51.059550 (+ 661us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:52:51.060324 (+ 774us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:52:51.060348 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:52:51.061399 (+ 1051us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:52:51.061404 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:52:51.063161 (+ 1757us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:52:51.063175 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:52:51.064004 (+ 829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:52:51.064010 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:52:51.064286 (+ 276us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:52:51.064969 (+ 683us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:52:51.064995 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:52:51.067570 (+ 2575us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:52:51.070387 (+ 2817us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:52:51.070395 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:52:51.070409 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:52:51.071023 (+ 614us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:52:51.071342 (+ 319us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:52:51.071346 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:52:51.071349 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:52:51.071572 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:52:51.072029 (+ 457us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:52:51.072038 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:52:51.072488 (+ 450us) client_negotiation.cc:769] Sending connection context
0809 22:52:51.072791 (+ 303us) client_negotiation.cc:240] Negotiation successful
0809 22:52:51.073308 (+ 517us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":551,"thread_start_us":317,"threads_started":1}
I20260809 22:52:51.076057 22047 heartbeater.cc:347] Connected to a master server at 127.15.113.190:37295
I20260809 22:52:51.076336 22047 heartbeater.cc:464] Registering TS with master...
I20260809 22:52:51.077142 22047 heartbeater.cc:511] Master 127.15.113.190:37295 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.15.113.131@KRBTEST.COM; defaulting to no policy
I20260809 22:52:51.078544 21503 ts_manager.cc:194] Registered new tserver with Master: b83de4e843eb425896ab5fc2cd0b41f6 (127.15.113.130:33259)
I20260809 22:52:51.079381 21503 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.113.130@KRBTEST.COM'} at 127.15.113.130:44709
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.113.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.113.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.131.keytab.
Entry for principal kudu/127.15.113.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.131.keytab.
I20260809 22:52:51.103885 15814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskItYPJR/build/asan/bin/kudu
/tmp/dist-test-taskItYPJR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315935617919-15814-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315935617919-15814-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315935617919-15814-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315935617919-15814-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.113.131:0
--local_ip_for_outbound_sockets=127.15.113.131
--webserver_interface=127.15.113.131
--webserver_port=0
--tserver_master_addrs=127.15.113.190:37295
--keytab_file=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.131.keytab
--principal=kudu/127.15.113.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.113.148:43167
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315935617919-15814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315935617919-15814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315935617919-15814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:52:51.416862 22057 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:52:51.417150 22057 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:52:51.417249 22057 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:52:51.424105 22057 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:52:51.424199 22057 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:52:51.424310 22057 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.113.131
I20260809 22:52:51.433614 22057 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.113.148:43167
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315935617919-15814-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315935617919-15814-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.131.keytab
--principal=kudu/127.15.113.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.113.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315935617919-15814-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.113.131
--webserver_port=0
--tserver_master_addrs=127.15.113.190:37295
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.113.131
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315935617919-15814-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:52:51.434708 22057 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:52:51.435930 22057 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:52:51.449594 22070 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:52:51.449638 22066 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:52:51.450143 22065 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:52:51.450549 22057 server_base.cc:1034] running on GCE node
I20260809 22:52:51.451159 22057 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:52:51.452028 22057 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:52:51.453214 22057 hybrid_clock.cc:648] HybridClock initialized: now 1786315971453159 us; error 39 us; skew 500 ppm
I20260809 22:52:51.453503 22057 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:52:51 dist-test-slave-ncp9 krb5kdc[21421](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315971, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.113.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:52:51.456364 22057 init.cc:377] Logged in from keytab as kudu/127.15.113.131@KRBTEST.COM (short username kudu)
I20260809 22:52:51.457724 22057 webserver.cc:460] Webserver started at http://127.15.113.131:42027/ using document root <none> and password file <none>
I20260809 22:52:51.458374 22057 fs_manager.cc:359] Metadata directory not provided
I20260809 22:52:51.458490 22057 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:52:51.458779 22057 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:52:51.461658 22057 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315935617919-15814-0/minicluster-data/ts-2/data/instance:
uuid: "86dc39ed0f1f4b6dad798f25a785b207"
format_stamp: "Formatted at 2026-08-09 22:52:51 on dist-test-slave-ncp9"
server_key: "1de3ebdb0b37b72695c20e01ef8cc604"
server_key_iv: "97f6333624afa6c9688d5500fd30ef24"
server_key_version: "encryptionkey@0"
I20260809 22:52:51.462427 22057 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315935617919-15814-0/minicluster-data/ts-2/wal/instance:
uuid: "86dc39ed0f1f4b6dad798f25a785b207"
format_stamp: "Formatted at 2026-08-09 22:52:51 on dist-test-slave-ncp9"
server_key: "1de3ebdb0b37b72695c20e01ef8cc604"
server_key_iv: "97f6333624afa6c9688d5500fd30ef24"
server_key_version: "encryptionkey@0"
I20260809 22:52:51.467276 22057 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 22:52:51.470589 22085 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:52:51.471773 22057 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260809 22:52:51.471925 22057 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315935617919-15814-0/minicluster-data/ts-2/data,/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315935617919-15814-0/minicluster-data/ts-2/wal
uuid: "86dc39ed0f1f4b6dad798f25a785b207"
format_stamp: "Formatted at 2026-08-09 22:52:51 on dist-test-slave-ncp9"
server_key: "1de3ebdb0b37b72695c20e01ef8cc604"
server_key_iv: "97f6333624afa6c9688d5500fd30ef24"
server_key_version: "encryptionkey@0"
I20260809 22:52:51.472105 22057 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315935617919-15814-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315935617919-15814-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315935617919-15814-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:52:51.492476 22057 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:52:51.496238 22057 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:52:51.496529 22057 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:52:51.497581 22057 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:52:51.498871 22057 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:52:51.498950 22057 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:52:51.499075 22057 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:52:51.499145 22057 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:52:51.529376 22057 rpc_server.cc:307] RPC server started. Bound to: 127.15.113.131:36949
I20260809 22:52:51.529424 22275 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.113.131:36949 every 8 connection(s)
I20260809 22:52:51.530706 22057 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315935617919-15814-0/minicluster-data/ts-2/data/info.pb
I20260809 22:52:51.531515 15814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskItYPJR/build/asan/bin/kudu as pid 22057
I20260809 22:52:51.531720 15814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315935617919-15814-0/minicluster-data/ts-2/wal/instance
I20260809 22:52:51.532246 15814 external_mini_cluster.cc:1428] Setting key 37c9c1f1211d9d0cbfe8242bc5a6ec2e
Aug 09 22:52:51 dist-test-slave-ncp9 krb5kdc[21421](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315971, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.113.131@KRBTEST.COM for kudu/127.15.113.190@KRBTEST.COM
I20260809 22:52:51.549891 21812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:52:51.533517 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.113.131:49515 (local address 127.15.113.190:37295)
0809 22:52:51.533712 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0809 22:52:51.533718 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:52:51.535181 (+ 1463us) server_negotiation.cc:407] Connection header received
0809 22:52:51.536961 (+ 1780us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:52:51.536965 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:52:51.537028 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:52:51.537105 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:52:51.538495 (+ 1390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:52:51.539208 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:52:51.540617 (+ 1409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:52:51.541049 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:52:51.541133 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:52:51.544226 (+ 3093us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:52:51.544231 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:52:51.544234 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:52:51.544263 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:52:51.547225 (+ 2962us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:52:51.547916 (+ 691us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:52:51.547920 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:52:51.547923 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:52:51.548063 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:52:51.548414 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:52:51.548417 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:52:51.548419 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:52:51.548696 (+ 277us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:52:51.548822 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0809 22:52:51.549449 (+ 627us) server_negotiation.cc:299] Negotiation successful
0809 22:52:51.549614 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260809 22:52:51.550756 22280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:52:51.534110 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:37295 (local address 127.15.113.131:49515)
0809 22:52:51.534997 (+ 887us) negotiation.cc:107] Waiting for socket to connect
0809 22:52:51.535060 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0809 22:52:51.536705 (+ 1645us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:52:51.537305 (+ 600us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:52:51.537313 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:52:51.537817 (+ 504us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:52:51.538363 (+ 546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:52:51.538380 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:52:51.539310 (+ 930us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:52:51.539314 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:52:51.540514 (+ 1200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:52:51.540525 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:52:51.541230 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:52:51.541235 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:52:51.541427 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:52:51.541984 (+ 557us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:52:51.542008 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:52:51.544031 (+ 2023us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:52:51.547358 (+ 3327us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:52:51.547368 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:52:51.547382 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:52:51.547808 (+ 426us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:52:51.548158 (+ 350us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:52:51.548161 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:52:51.548163 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:52:51.548334 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:52:51.548795 (+ 461us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:52:51.548800 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:52:51.549137 (+ 337us) client_negotiation.cc:769] Sending connection context
0809 22:52:51.549356 (+ 219us) client_negotiation.cc:240] Negotiation successful
0809 22:52:51.549702 (+ 346us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":584,"thread_start_us":330,"threads_started":1}
I20260809 22:52:51.552053 22276 heartbeater.cc:347] Connected to a master server at 127.15.113.190:37295
I20260809 22:52:51.552325 22276 heartbeater.cc:464] Registering TS with master...
I20260809 22:52:51.553092 22276 heartbeater.cc:511] Master 127.15.113.190:37295 requested a full tablet report, sending...
I20260809 22:52:51.554574 21503 ts_manager.cc:194] Registered new tserver with Master: 86dc39ed0f1f4b6dad798f25a785b207 (127.15.113.131:36949)
I20260809 22:52:51.555418 21503 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.113.131@KRBTEST.COM'} at 127.15.113.131:49515
I20260809 22:52:51.559779 15814 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 22:52:51.585427 21809 heartbeater.cc:503] Master 127.15.113.190:37295 was elected leader, sending a full tablet report...
Aug 09 22:52:51 dist-test-slave-ncp9 krb5kdc[21421](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315971, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 09 22:52:51 dist-test-slave-ncp9 krb5kdc[21421](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315971, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.113.129@KRBTEST.COM
I20260809 22:52:51.612736 22287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:52:51.598306 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58324 (local address 127.15.113.129:35591)
0809 22:52:51.599131 (+ 825us) server_negotiation.cc:206] Beginning negotiation
0809 22:52:51.599138 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:52:51.599165 (+ 27us) server_negotiation.cc:407] Connection header received
0809 22:52:51.599224 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:52:51.599231 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:52:51.599467 (+ 236us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:52:51.599602 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:52:51.601222 (+ 1620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:52:51.602356 (+ 1134us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:52:51.603495 (+ 1139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:52:51.604059 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:52:51.604152 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:52:51.607695 (+ 3543us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:52:51.607706 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:52:51.607721 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:52:51.607761 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:52:51.610552 (+ 2791us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:52:51.611081 (+ 529us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:52:51.611087 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:52:51.611089 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:52:51.611212 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:52:51.611484 (+ 272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:52:51.611487 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:52:51.611489 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:52:51.611826 (+ 337us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:52:51.612049 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0809 22:52:51.612255 (+ 206us) server_negotiation.cc:299] Negotiation successful
0809 22:52:51.612447 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":676,"thread_start_us":476,"threads_started":1}
Aug 09 22:52:51 dist-test-slave-ncp9 krb5kdc[21421](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315971, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.113.190@KRBTEST.COM
I20260809 22:52:51.628696 21812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:52:51.616668 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33398 (local address 127.15.113.190:37295)
0809 22:52:51.616868 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0809 22:52:51.616874 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:52:51.617070 (+ 196us) server_negotiation.cc:407] Connection header received
0809 22:52:51.617183 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:52:51.617187 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:52:51.617248 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:52:51.617323 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:52:51.618409 (+ 1086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:52:51.619111 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:52:51.620232 (+ 1121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:52:51.620626 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:52:51.620724 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:52:51.624090 (+ 3366us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:52:51.624095 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:52:51.624099 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:52:51.624133 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:52:51.626579 (+ 2446us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:52:51.627137 (+ 558us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:52:51.627140 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:52:51.627142 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:52:51.627271 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:52:51.627528 (+ 257us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:52:51.627531 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:52:51.627533 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:52:51.627828 (+ 295us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:52:51.628021 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0809 22:52:51.628230 (+ 209us) server_negotiation.cc:299] Negotiation successful
0809 22:52:51.628406 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
W20260809 22:52:51.629251 21503 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33398
I20260809 22:52:51.630131 15814 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskItYPJR/build/asan/bin/kudu pid 21593
I20260809 22:52:51.642009 22287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:52:51.631094 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58334 (local address 127.15.113.129:35591)
0809 22:52:51.631288 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0809 22:52:51.631294 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:52:51.631410 (+ 116us) server_negotiation.cc:407] Connection header received
0809 22:52:51.631574 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:52:51.631579 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:52:51.631653 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:52:51.631737 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:52:51.632939 (+ 1202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:52:51.633614 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:52:51.634819 (+ 1205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:52:51.635259 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:52:51.635364 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:52:51.637430 (+ 2066us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:52:51.637435 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:52:51.637439 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:52:51.637469 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:52:51.639645 (+ 2176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:52:51.640219 (+ 574us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:52:51.640224 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:52:51.640228 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:52:51.640351 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:52:51.640693 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:52:51.640698 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:52:51.640701 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:52:51.640979 (+ 278us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:52:51.641201 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0809 22:52:51.641500 (+ 299us) server_negotiation.cc:299] Negotiation successful
0809 22:52:51.641724 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260809 22:52:51.642657 21800 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:52:51.724720 15814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskItYPJR/build/asan/bin/kudu with pid 21593
I20260809 22:52:51.749544 15814 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskItYPJR/build/asan/bin/kudu pid 21818
Aug 09 22:52:51 dist-test-slave-ncp9 krb5kdc[21421](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315971, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.113.130@KRBTEST.COM
I20260809 22:52:51.764197 22296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:52:51.750457 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45612 (local address 127.15.113.130:33259)
0809 22:52:51.751005 (+ 548us) server_negotiation.cc:206] Beginning negotiation
0809 22:52:51.751013 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 22:52:51.751039 (+ 26us) server_negotiation.cc:407] Connection header received
0809 22:52:51.751099 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:52:51.751105 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:52:51.751328 (+ 223us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:52:51.751455 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:52:51.752561 (+ 1106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:52:51.753615 (+ 1054us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:52:51.754809 (+ 1194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:52:51.755359 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:52:51.755452 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:52:51.759202 (+ 3750us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:52:51.759213 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:52:51.759227 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:52:51.759259 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:52:51.761929 (+ 2670us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:52:51.762510 (+ 581us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:52:51.762516 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:52:51.762519 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:52:51.762633 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:52:51.762900 (+ 267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:52:51.762903 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:52:51.762905 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:52:51.763230 (+ 325us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:52:51.763448 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0809 22:52:51.763655 (+ 207us) server_negotiation.cc:299] Negotiation successful
0809 22:52:51.763838 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":416,"thread_start_us":285,"threads_started":1}
I20260809 22:52:51.765528 22037 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:52:51.844954 15814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskItYPJR/build/asan/bin/kudu with pid 21818
I20260809 22:52:51.868846 15814 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskItYPJR/build/asan/bin/kudu pid 22057
Aug 09 22:52:51 dist-test-slave-ncp9 krb5kdc[21421](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315971, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.113.131@KRBTEST.COM
I20260809 22:52:51.884789 22298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:52:51.869801 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39836 (local address 127.15.113.131:36949)
0809 22:52:51.870456 (+ 655us) server_negotiation.cc:206] Beginning negotiation
0809 22:52:51.870464 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 22:52:51.870493 (+ 29us) server_negotiation.cc:407] Connection header received
0809 22:52:51.870554 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:52:51.870560 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:52:51.870776 (+ 216us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:52:51.870933 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:52:51.872209 (+ 1276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:52:51.873362 (+ 1153us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:52:51.874645 (+ 1283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:52:51.875259 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:52:51.875371 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:52:51.879542 (+ 4171us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:52:51.879554 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:52:51.879568 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:52:51.879608 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:52:51.882145 (+ 2537us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:52:51.882789 (+ 644us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:52:51.882796 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:52:51.882798 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:52:51.882921 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:52:51.883260 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:52:51.883264 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:52:51.883266 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:52:51.883608 (+ 342us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:52:51.883893 (+ 285us) server_negotiation.cc:1035] Waiting for connection context
0809 22:52:51.884201 (+ 308us) server_negotiation.cc:299] Negotiation successful
0809 22:52:51.884463 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":496,"thread_start_us":326,"threads_started":1}
I20260809 22:52:51.886067 22268 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:52:51.969410 15814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskItYPJR/build/asan/bin/kudu with pid 22057
I20260809 22:52:51.987473 15814 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskItYPJR/build/asan/bin/kudu pid 21451
I20260809 22:52:51.988629 21548 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:52:52.091578 15814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskItYPJR/build/asan/bin/kudu with pid 21451
2026-08-09T22:52:52Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (3948 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315935617919-15814-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 22:52:52 dist-test-slave-ncp9 krb5kdc[22306](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 22:52:52 dist-test-slave-ncp9 krb5kdc[22306](info): set up 2 sockets
Aug 09 22:52:52 dist-test-slave-ncp9 krb5kdc[22306](info): commencing operation
krb5kdc: starting...
W20260809 22:52:53.285218 15814 mini_kdc.cc:121] Time spent starting KDC: real 1.144s user 0.000s sys 0.027s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315935617919-15814-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315935617919-15814-0/krb5kdc/test-user.keytab.
Aug 09 22:52:53 dist-test-slave-ncp9 krb5kdc[22306](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315973, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T22:52:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T22:52:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.113.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.113.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.113.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.190.keytab.
Entry for principal kudu/127.15.113.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.190.keytab.
I20260809 22:52:53.490067 15814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskItYPJR/build/asan/bin/kudu
/tmp/dist-test-taskItYPJR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315935617919-15814-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315935617919-15814-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315935617919-15814-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315935617919-15814-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.113.190:37153
--webserver_interface=127.15.113.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.190.keytab
--principal=kudu/127.15.113.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.113.148:34285
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.113.190:37153
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315935617919-15814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315935617919-15814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315935617919-15814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:52:53.812144 22330 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:52:53.812436 22330 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:52:53.812578 22330 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:52:53.819509 22330 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 22:52:53.819617 22330 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:52:53.819695 22330 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:52:53.819772 22330 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 22:52:53.819837 22330 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 22:52:53.829286 22330 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.113.148:34285
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315935617919-15814-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315935617919-15814-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.113.190:37153
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.190.keytab
--principal=kudu/127.15.113.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.113.190:37153
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315935617919-15814-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.113.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315935617919-15814-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:52:53.830498 22330 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:52:53.831728 22330 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:52:53.842530 22344 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:52:53.842571 22342 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:52:53.842490 22341 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:52:53.843503 22330 server_base.cc:1034] running on GCE node
I20260809 22:52:53.844213 22330 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:52:53.845531 22330 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:52:53.846719 22330 hybrid_clock.cc:648] HybridClock initialized: now 1786315973846671 us; error 44 us; skew 500 ppm
I20260809 22:52:53.847023 22330 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:52:53 dist-test-slave-ncp9 krb5kdc[22306](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315973, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.113.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:52:53.849888 22330 init.cc:377] Logged in from keytab as kudu/127.15.113.190@KRBTEST.COM (short username kudu)
I20260809 22:52:53.850984 22330 fs_manager.cc:359] Metadata directory not provided
I20260809 22:52:53.851091 22330 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:52:53.851394 22330 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:52:53.854234 22330 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315935617919-15814-0/minicluster-data/master-0/data/instance:
uuid: "d9d8c645b1514b3ebab412b28bae9cfa"
format_stamp: "Formatted at 2026-08-09 22:52:53 on dist-test-slave-ncp9"
server_key: "6644d673570ec1262fb7072aa4e6c16d"
server_key_iv: "80337ab587d99a5e55d03ab282ee8441"
server_key_version: "encryptionkey@0"
I20260809 22:52:53.854984 22330 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315935617919-15814-0/minicluster-data/master-0/wal/instance:
uuid: "d9d8c645b1514b3ebab412b28bae9cfa"
format_stamp: "Formatted at 2026-08-09 22:52:53 on dist-test-slave-ncp9"
server_key: "6644d673570ec1262fb7072aa4e6c16d"
server_key_iv: "80337ab587d99a5e55d03ab282ee8441"
server_key_version: "encryptionkey@0"
I20260809 22:52:53.859648 22330 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.001s
I20260809 22:52:53.862895 22358 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:52:53.864019 22330 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 22:52:53.864187 22330 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315935617919-15814-0/minicluster-data/master-0/data,/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315935617919-15814-0/minicluster-data/master-0/wal
uuid: "d9d8c645b1514b3ebab412b28bae9cfa"
format_stamp: "Formatted at 2026-08-09 22:52:53 on dist-test-slave-ncp9"
server_key: "6644d673570ec1262fb7072aa4e6c16d"
server_key_iv: "80337ab587d99a5e55d03ab282ee8441"
server_key_version: "encryptionkey@0"
I20260809 22:52:53.864390 22330 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315935617919-15814-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315935617919-15814-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315935617919-15814-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:52:53.880326 22330 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:52:53.883801 22330 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:52:53.884030 22330 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:52:53.900957 22330 rpc_server.cc:307] RPC server started. Bound to: 127.15.113.190:37153
I20260809 22:52:53.900976 22438 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.113.190:37153 every 8 connection(s)
I20260809 22:52:53.901825 22330 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315935617919-15814-0/minicluster-data/master-0/data/info.pb
I20260809 22:52:53.905315 22439 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:52:53.907600 15814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskItYPJR/build/asan/bin/kudu as pid 22330
I20260809 22:52:53.907797 15814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315935617919-15814-0/minicluster-data/master-0/wal/instance
I20260809 22:52:53.908416 15814 external_mini_cluster.cc:1428] Setting key 4c6efc597d24eb0c059d2d008ecceb47
I20260809 22:52:53.917645 22439 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d9d8c645b1514b3ebab412b28bae9cfa: Bootstrap starting.
I20260809 22:52:53.921104 22439 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d9d8c645b1514b3ebab412b28bae9cfa: Neither blocks nor log segments found. Creating new log.
I20260809 22:52:53.922039 22439 log.cc:824] T 00000000000000000000000000000000 P d9d8c645b1514b3ebab412b28bae9cfa: Log is configured to *not* fsync() on all Append() calls
Aug 09 22:52:53 dist-test-slave-ncp9 krb5kdc[22306](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315973, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.113.190@KRBTEST.COM
I20260809 22:52:53.924813 22439 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d9d8c645b1514b3ebab412b28bae9cfa: No bootstrap required, opened a new log
I20260809 22:52:53.932300 22444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:52:53.910434 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42032 (local address 127.15.113.190:37153)
0809 22:52:53.911275 (+ 841us) server_negotiation.cc:206] Beginning negotiation
0809 22:52:53.911289 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0809 22:52:53.911335 (+ 46us) server_negotiation.cc:407] Connection header received
0809 22:52:53.913555 (+ 2220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:52:53.913581 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:52:53.914074 (+ 493us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:52:53.914502 (+ 428us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:52:53.915784 (+ 1282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:52:53.917697 (+ 1913us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:52:53.918981 (+ 1284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:52:53.920011 (+ 1030us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:52:53.920183 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:52:53.924067 (+ 3884us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:52:53.924078 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:52:53.924096 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:52:53.924142 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:52:53.927952 (+ 3810us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:52:53.928594 (+ 642us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:52:53.928602 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:52:53.928610 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:52:53.928816 (+ 206us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:52:53.929187 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:52:53.929192 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:52:53.929196 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:52:53.929891 (+ 695us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:52:53.930189 (+ 298us) server_negotiation.cc:1035] Waiting for connection context
0809 22:52:53.930560 (+ 371us) server_negotiation.cc:299] Negotiation successful
0809 22:52:53.931003 (+ 443us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":592,"thread_start_us":351,"threads_started":1}
I20260809 22:52:53.934515 22439 raft_consensus.cc:348] T 00000000000000000000000000000000 P d9d8c645b1514b3ebab412b28bae9cfa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d9d8c645b1514b3ebab412b28bae9cfa" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 37153 } }
I20260809 22:52:53.934851 22439 raft_consensus.cc:374] T 00000000000000000000000000000000 P d9d8c645b1514b3ebab412b28bae9cfa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:52:53.935006 22439 raft_consensus.cc:729] T 00000000000000000000000000000000 P d9d8c645b1514b3ebab412b28bae9cfa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d9d8c645b1514b3ebab412b28bae9cfa, State: Initialized, Role: FOLLOWER
I20260809 22:52:53.935544 22439 consensus_queue.cc:260] T 00000000000000000000000000000000 P d9d8c645b1514b3ebab412b28bae9cfa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d9d8c645b1514b3ebab412b28bae9cfa" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 37153 } }
I20260809 22:52:53.935745 22439 raft_consensus.cc:388] T 00000000000000000000000000000000 P d9d8c645b1514b3ebab412b28bae9cfa [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 22:52:53.935837 22439 raft_consensus.cc:482] T 00000000000000000000000000000000 P d9d8c645b1514b3ebab412b28bae9cfa [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 22:52:53.935956 22439 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d9d8c645b1514b3ebab412b28bae9cfa [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:52:53.938020 22439 raft_consensus.cc:504] T 00000000000000000000000000000000 P d9d8c645b1514b3ebab412b28bae9cfa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d9d8c645b1514b3ebab412b28bae9cfa" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 37153 } }
I20260809 22:52:53.938482 22439 leader_election.cc:302] T 00000000000000000000000000000000 P d9d8c645b1514b3ebab412b28bae9cfa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d9d8c645b1514b3ebab412b28bae9cfa; no voters:
I20260809 22:52:53.939105 22439 leader_election.cc:288] T 00000000000000000000000000000000 P d9d8c645b1514b3ebab412b28bae9cfa [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 22:52:53.939263 22447 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d9d8c645b1514b3ebab412b28bae9cfa [term 1 FOLLOWER]: Leader election won for term 1
I20260809 22:52:53.939906 22447 raft_consensus.cc:686] T 00000000000000000000000000000000 P d9d8c645b1514b3ebab412b28bae9cfa [term 1 LEADER]: Becoming Leader. State: Replica: d9d8c645b1514b3ebab412b28bae9cfa, State: Running, Role: LEADER
I20260809 22:52:53.940397 22447 consensus_queue.cc:237] T 00000000000000000000000000000000 P d9d8c645b1514b3ebab412b28bae9cfa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d9d8c645b1514b3ebab412b28bae9cfa" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 37153 } }
I20260809 22:52:53.941061 22439 sys_catalog.cc:564] T 00000000000000000000000000000000 P d9d8c645b1514b3ebab412b28bae9cfa [sys.catalog]: configured and running, proceeding with master startup.
I20260809 22:52:53.942909 22448 sys_catalog.cc:455] T 00000000000000000000000000000000 P d9d8c645b1514b3ebab412b28bae9cfa [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d9d8c645b1514b3ebab412b28bae9cfa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d9d8c645b1514b3ebab412b28bae9cfa" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 37153 } } }
I20260809 22:52:53.943516 22448 sys_catalog.cc:458] T 00000000000000000000000000000000 P d9d8c645b1514b3ebab412b28bae9cfa [sys.catalog]: This master's current role is: LEADER
I20260809 22:52:53.943321 22449 sys_catalog.cc:455] T 00000000000000000000000000000000 P d9d8c645b1514b3ebab412b28bae9cfa [sys.catalog]: SysCatalogTable state changed. Reason: New leader d9d8c645b1514b3ebab412b28bae9cfa. Latest consensus state: current_term: 1 leader_uuid: "d9d8c645b1514b3ebab412b28bae9cfa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d9d8c645b1514b3ebab412b28bae9cfa" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 37153 } } }
I20260809 22:52:53.943652 22449 sys_catalog.cc:458] T 00000000000000000000000000000000 P d9d8c645b1514b3ebab412b28bae9cfa [sys.catalog]: This master's current role is: LEADER
I20260809 22:52:53.945225 22461 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 22:52:53.949630 22461 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 22:52:53.957227 22461 catalog_manager.cc:1347] Generated new cluster ID: 5ba09999c3df41d8ba2588c8030a673e
I20260809 22:52:53.957333 22461 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 22:52:53.970253 22461 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 22:52:53.971537 22461 catalog_manager.cc:1504] Loading token signing keys...
I20260809 22:52:53.982875 22461 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d9d8c645b1514b3ebab412b28bae9cfa: Generated new TSK 0
I20260809 22:52:53.983685 22461 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.113.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.113.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.113.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.129.keytab.
Entry for principal kudu/127.15.113.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.129.keytab.
I20260809 22:52:54.033758 15814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskItYPJR/build/asan/bin/kudu
/tmp/dist-test-taskItYPJR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315935617919-15814-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315935617919-15814-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315935617919-15814-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315935617919-15814-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.113.129:0
--local_ip_for_outbound_sockets=127.15.113.129
--webserver_interface=127.15.113.129
--webserver_port=0
--tserver_master_addrs=127.15.113.190:37153
--keytab_file=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.129.keytab
--principal=kudu/127.15.113.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.113.148:34285
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315935617919-15814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315935617919-15814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315935617919-15814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:52:54.357020 22479 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:52:54.357270 22479 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:52:54.357422 22479 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:52:54.364820 22479 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:52:54.364943 22479 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:52:54.365123 22479 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.113.129
I20260809 22:52:54.374643 22479 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.113.148:34285
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315935617919-15814-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315935617919-15814-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.129.keytab
--principal=kudu/127.15.113.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.113.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315935617919-15814-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.113.129
--webserver_port=0
--tserver_master_addrs=127.15.113.190:37153
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.113.129
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315935617919-15814-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:52:54.375770 22479 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:52:54.377012 22479 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:52:54.389222 22491 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:52:54.389294 22493 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:52:54.389329 22496 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:52:54.389972 22479 server_base.cc:1034] running on GCE node
I20260809 22:52:54.390522 22479 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:52:54.391342 22479 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:52:54.392534 22479 hybrid_clock.cc:648] HybridClock initialized: now 1786315974392494 us; error 32 us; skew 500 ppm
I20260809 22:52:54.392823 22479 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:52:54 dist-test-slave-ncp9 krb5kdc[22306](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315974, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.113.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:52:54.395757 22479 init.cc:377] Logged in from keytab as kudu/127.15.113.129@KRBTEST.COM (short username kudu)
I20260809 22:52:54.396740 22479 fs_manager.cc:359] Metadata directory not provided
I20260809 22:52:54.396828 22479 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:52:54.397138 22479 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:52:54.399885 22479 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315935617919-15814-0/minicluster-data/ts-0/data/instance:
uuid: "96a6a862316c48f3bdb1b24bc6c8037a"
format_stamp: "Formatted at 2026-08-09 22:52:54 on dist-test-slave-ncp9"
server_key: "665dfda4070380690a7267bdf2a2aeb7"
server_key_iv: "fb6d26037579ec85cc0aaa4db9d11460"
server_key_version: "encryptionkey@0"
I20260809 22:52:54.400619 22479 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315935617919-15814-0/minicluster-data/ts-0/wal/instance:
uuid: "96a6a862316c48f3bdb1b24bc6c8037a"
format_stamp: "Formatted at 2026-08-09 22:52:54 on dist-test-slave-ncp9"
server_key: "665dfda4070380690a7267bdf2a2aeb7"
server_key_iv: "fb6d26037579ec85cc0aaa4db9d11460"
server_key_version: "encryptionkey@0"
I20260809 22:52:54.405463 22479 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260809 22:52:54.408558 22507 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:52:54.409756 22479 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 22:52:54.409899 22479 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315935617919-15814-0/minicluster-data/ts-0/data,/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315935617919-15814-0/minicluster-data/ts-0/wal
uuid: "96a6a862316c48f3bdb1b24bc6c8037a"
format_stamp: "Formatted at 2026-08-09 22:52:54 on dist-test-slave-ncp9"
server_key: "665dfda4070380690a7267bdf2a2aeb7"
server_key_iv: "fb6d26037579ec85cc0aaa4db9d11460"
server_key_version: "encryptionkey@0"
I20260809 22:52:54.410081 22479 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315935617919-15814-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315935617919-15814-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315935617919-15814-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:52:54.430281 22479 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:52:54.433977 22479 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:52:54.434221 22479 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:52:54.435115 22479 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:52:54.436378 22479 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:52:54.436465 22479 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:52:54.436568 22479 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:52:54.436640 22479 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:52:54.466173 22479 rpc_server.cc:307] RPC server started. Bound to: 127.15.113.129:45677
I20260809 22:52:54.466224 22693 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.113.129:45677 every 8 connection(s)
I20260809 22:52:54.466952 22479 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315935617919-15814-0/minicluster-data/ts-0/data/info.pb
I20260809 22:52:54.473263 15814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskItYPJR/build/asan/bin/kudu as pid 22479
I20260809 22:52:54.473459 15814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315935617919-15814-0/minicluster-data/ts-0/wal/instance
I20260809 22:52:54.474064 15814 external_mini_cluster.cc:1428] Setting key 4c77d78e2d29aa4320584d97d888849d
Aug 09 22:52:54 dist-test-slave-ncp9 krb5kdc[22306](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315974, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.113.129@KRBTEST.COM for kudu/127.15.113.190@KRBTEST.COM
I20260809 22:52:54.486182 22697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:52:54.469333 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.113.129:32817 (local address 127.15.113.190:37153)
0809 22:52:54.469754 (+ 421us) server_negotiation.cc:206] Beginning negotiation
0809 22:52:54.469759 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:52:54.470582 (+ 823us) server_negotiation.cc:407] Connection header received
0809 22:52:54.472387 (+ 1805us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:52:54.472393 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:52:54.472472 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:52:54.472566 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:52:54.474222 (+ 1656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:52:54.475063 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:52:54.476667 (+ 1604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:52:54.477096 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:52:54.477202 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:52:54.480447 (+ 3245us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:52:54.480452 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:52:54.480455 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:52:54.480486 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:52:54.483266 (+ 2780us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:52:54.483943 (+ 677us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:52:54.483947 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:52:54.483949 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:52:54.484067 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:52:54.484460 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:52:54.484464 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:52:54.484466 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:52:54.484735 (+ 269us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:52:54.484884 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0809 22:52:54.485578 (+ 694us) server_negotiation.cc:299] Negotiation successful
0809 22:52:54.485811 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":318,"thread_start_us":223,"threads_started":1}
I20260809 22:52:54.487092 22699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:52:54.469777 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:37153 (local address 127.15.113.129:32817)
0809 22:52:54.470440 (+ 663us) negotiation.cc:107] Waiting for socket to connect
0809 22:52:54.470477 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0809 22:52:54.472097 (+ 1620us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:52:54.472813 (+ 716us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:52:54.472821 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:52:54.473432 (+ 611us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:52:54.474020 (+ 588us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:52:54.474040 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:52:54.475208 (+ 1168us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:52:54.475212 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:52:54.476518 (+ 1306us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:52:54.476531 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:52:54.477333 (+ 802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:52:54.477340 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:52:54.477557 (+ 217us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:52:54.478168 (+ 611us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:52:54.478189 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:52:54.480243 (+ 2054us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:52:54.483403 (+ 3160us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:52:54.483409 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:52:54.483420 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:52:54.483829 (+ 409us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:52:54.484173 (+ 344us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:52:54.484177 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:52:54.484179 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:52:54.484350 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:52:54.484884 (+ 534us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:52:54.484913 (+ 29us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:52:54.485230 (+ 317us) client_negotiation.cc:769] Sending connection context
0809 22:52:54.485468 (+ 238us) client_negotiation.cc:240] Negotiation successful
0809 22:52:54.485967 (+ 499us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":445,"thread_start_us":277,"threads_started":1}
I20260809 22:52:54.488541 22694 heartbeater.cc:347] Connected to a master server at 127.15.113.190:37153
I20260809 22:52:54.488817 22694 heartbeater.cc:464] Registering TS with master...
I20260809 22:52:54.489449 22694 heartbeater.cc:511] Master 127.15.113.190:37153 requested a full tablet report, sending...
I20260809 22:52:54.491586 22388 ts_manager.cc:194] Registered new tserver with Master: 96a6a862316c48f3bdb1b24bc6c8037a (127.15.113.129:45677)
I20260809 22:52:54.493541 22388 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.113.129@KRBTEST.COM'} at 127.15.113.129:32817
WARNING: no policy specified for kudu/127.15.113.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.113.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.113.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.130.keytab.
Entry for principal kudu/127.15.113.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.130.keytab.
I20260809 22:52:54.524796 15814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskItYPJR/build/asan/bin/kudu
/tmp/dist-test-taskItYPJR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315935617919-15814-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315935617919-15814-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315935617919-15814-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315935617919-15814-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.113.130:0
--local_ip_for_outbound_sockets=127.15.113.130
--webserver_interface=127.15.113.130
--webserver_port=0
--tserver_master_addrs=127.15.113.190:37153
--keytab_file=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.130.keytab
--principal=kudu/127.15.113.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.113.148:34285
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315935617919-15814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315935617919-15814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315935617919-15814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:52:54.843737 22703 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:52:54.843992 22703 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:52:54.844094 22703 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:52:54.850973 22703 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:52:54.851068 22703 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:52:54.851224 22703 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.113.130
I20260809 22:52:54.860355 22703 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.113.148:34285
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315935617919-15814-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315935617919-15814-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.130.keytab
--principal=kudu/127.15.113.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.113.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315935617919-15814-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.113.130
--webserver_port=0
--tserver_master_addrs=127.15.113.190:37153
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.113.130
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315935617919-15814-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:52:54.861590 22703 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:52:54.862831 22703 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:52:54.875068 22717 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:52:54.875008 22713 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:52:54.875399 22711 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:52:54.875746 22703 server_base.cc:1034] running on GCE node
I20260809 22:52:54.876317 22703 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:52:54.877184 22703 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:52:54.878381 22703 hybrid_clock.cc:648] HybridClock initialized: now 1786315974878328 us; error 46 us; skew 500 ppm
I20260809 22:52:54.878669 22703 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:52:54 dist-test-slave-ncp9 krb5kdc[22306](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315974, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.113.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:52:54.881692 22703 init.cc:377] Logged in from keytab as kudu/127.15.113.130@KRBTEST.COM (short username kudu)
I20260809 22:52:54.882726 22703 fs_manager.cc:359] Metadata directory not provided
I20260809 22:52:54.882825 22703 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:52:54.883108 22703 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:52:54.885905 22703 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315935617919-15814-0/minicluster-data/ts-1/data/instance:
uuid: "758af62e22834fdfb9e8604c7f28a075"
format_stamp: "Formatted at 2026-08-09 22:52:54 on dist-test-slave-ncp9"
server_key: "5150bcbaa194d278c80dd37538951e44"
server_key_iv: "3c8f26dd2b928c42765ca85b3a7f4fbc"
server_key_version: "encryptionkey@0"
I20260809 22:52:54.886689 22703 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315935617919-15814-0/minicluster-data/ts-1/wal/instance:
uuid: "758af62e22834fdfb9e8604c7f28a075"
format_stamp: "Formatted at 2026-08-09 22:52:54 on dist-test-slave-ncp9"
server_key: "5150bcbaa194d278c80dd37538951e44"
server_key_iv: "3c8f26dd2b928c42765ca85b3a7f4fbc"
server_key_version: "encryptionkey@0"
I20260809 22:52:54.891737 22703 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260809 22:52:54.895087 22729 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:52:54.896303 22703 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.001s
I20260809 22:52:54.896459 22703 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315935617919-15814-0/minicluster-data/ts-1/data,/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315935617919-15814-0/minicluster-data/ts-1/wal
uuid: "758af62e22834fdfb9e8604c7f28a075"
format_stamp: "Formatted at 2026-08-09 22:52:54 on dist-test-slave-ncp9"
server_key: "5150bcbaa194d278c80dd37538951e44"
server_key_iv: "3c8f26dd2b928c42765ca85b3a7f4fbc"
server_key_version: "encryptionkey@0"
I20260809 22:52:54.896652 22703 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315935617919-15814-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315935617919-15814-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315935617919-15814-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:52:54.921005 22703 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:52:54.925051 22703 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:52:54.925297 22703 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:52:54.926190 22703 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:52:54.927464 22703 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:52:54.927567 22703 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:52:54.927668 22703 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:52:54.927738 22703 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:52:54.957747 22703 rpc_server.cc:307] RPC server started. Bound to: 127.15.113.130:40603
I20260809 22:52:54.957808 22899 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.113.130:40603 every 8 connection(s)
I20260809 22:52:54.958554 22703 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315935617919-15814-0/minicluster-data/ts-1/data/info.pb
I20260809 22:52:54.963882 15814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskItYPJR/build/asan/bin/kudu as pid 22703
I20260809 22:52:54.964083 15814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315935617919-15814-0/minicluster-data/ts-1/wal/instance
I20260809 22:52:54.964543 15814 external_mini_cluster.cc:1428] Setting key 7b7a96908bbef852e227f95f12bf346e
Aug 09 22:52:54 dist-test-slave-ncp9 krb5kdc[22306](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315974, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.113.130@KRBTEST.COM for kudu/127.15.113.190@KRBTEST.COM
I20260809 22:52:54.977988 22697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:52:54.960831 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.113.130:34879 (local address 127.15.113.190:37153)
0809 22:52:54.961022 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0809 22:52:54.961026 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 22:52:54.962149 (+ 1123us) server_negotiation.cc:407] Connection header received
0809 22:52:54.963915 (+ 1766us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:52:54.963921 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:52:54.964012 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:52:54.964140 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:52:54.965649 (+ 1509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:52:54.966489 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:52:54.968105 (+ 1616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:52:54.968759 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:52:54.968877 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:52:54.972111 (+ 3234us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:52:54.972117 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:52:54.972129 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:52:54.972166 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:52:54.975056 (+ 2890us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:52:54.975738 (+ 682us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:52:54.975742 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:52:54.975745 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:52:54.975873 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:52:54.976284 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:52:54.976288 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:52:54.976290 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:52:54.976555 (+ 265us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:52:54.976697 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0809 22:52:54.977430 (+ 733us) server_negotiation.cc:299] Negotiation successful
0809 22:52:54.977665 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260809 22:52:54.978938 22907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:52:54.961299 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:37153 (local address 127.15.113.130:34879)
0809 22:52:54.961986 (+ 687us) negotiation.cc:107] Waiting for socket to connect
0809 22:52:54.962024 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0809 22:52:54.963644 (+ 1620us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:52:54.964340 (+ 696us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:52:54.964348 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:52:54.964875 (+ 527us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:52:54.965476 (+ 601us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:52:54.965502 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:52:54.966642 (+ 1140us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:52:54.966646 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:52:54.967954 (+ 1308us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:52:54.967968 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:52:54.968981 (+ 1013us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:52:54.968988 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:52:54.969217 (+ 229us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:52:54.969832 (+ 615us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:52:54.969854 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:52:54.971903 (+ 2049us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:52:54.975184 (+ 3281us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:52:54.975191 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:52:54.975203 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:52:54.975608 (+ 405us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:52:54.975976 (+ 368us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:52:54.975980 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:52:54.975982 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:52:54.976170 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:52:54.976677 (+ 507us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:52:54.976685 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:52:54.977060 (+ 375us) client_negotiation.cc:769] Sending connection context
0809 22:52:54.977303 (+ 243us) client_negotiation.cc:240] Negotiation successful
0809 22:52:54.977834 (+ 531us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":457,"thread_start_us":275,"threads_started":1}
I20260809 22:52:54.980439 22903 heartbeater.cc:347] Connected to a master server at 127.15.113.190:37153
I20260809 22:52:54.980780 22903 heartbeater.cc:464] Registering TS with master...
I20260809 22:52:54.981637 22903 heartbeater.cc:511] Master 127.15.113.190:37153 requested a full tablet report, sending...
I20260809 22:52:54.983315 22388 ts_manager.cc:194] Registered new tserver with Master: 758af62e22834fdfb9e8604c7f28a075 (127.15.113.130:40603)
I20260809 22:52:54.984359 22388 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.113.130@KRBTEST.COM'} at 127.15.113.130:34879
WARNING: no policy specified for kudu/127.15.113.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.113.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.113.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.131.keytab.
Entry for principal kudu/127.15.113.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.131.keytab.
I20260809 22:52:55.016031 15814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskItYPJR/build/asan/bin/kudu
/tmp/dist-test-taskItYPJR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315935617919-15814-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315935617919-15814-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315935617919-15814-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315935617919-15814-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.113.131:0
--local_ip_for_outbound_sockets=127.15.113.131
--webserver_interface=127.15.113.131
--webserver_port=0
--tserver_master_addrs=127.15.113.190:37153
--keytab_file=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.131.keytab
--principal=kudu/127.15.113.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.113.148:34285
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315935617919-15814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315935617919-15814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315935617919-15814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:52:55.329018 22910 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:52:55.329268 22910 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:52:55.329366 22910 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:52:55.336292 22910 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:52:55.336395 22910 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:52:55.336509 22910 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.113.131
I20260809 22:52:55.345856 22910 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.113.148:34285
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315935617919-15814-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315935617919-15814-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.131.keytab
--principal=kudu/127.15.113.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.113.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315935617919-15814-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.113.131
--webserver_port=0
--tserver_master_addrs=127.15.113.190:37153
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.113.131
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315935617919-15814-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:52:55.347015 22910 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:52:55.348217 22910 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:52:55.360173 22921 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:52:55.360234 22924 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:52:55.360234 22920 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:52:55.361268 22910 server_base.cc:1034] running on GCE node
I20260809 22:52:55.361809 22910 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:52:55.362623 22910 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:52:55.363833 22910 hybrid_clock.cc:648] HybridClock initialized: now 1786315975363781 us; error 41 us; skew 500 ppm
I20260809 22:52:55.364135 22910 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:52:55 dist-test-slave-ncp9 krb5kdc[22306](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315975, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.113.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:52:55.366927 22910 init.cc:377] Logged in from keytab as kudu/127.15.113.131@KRBTEST.COM (short username kudu)
I20260809 22:52:55.368127 22910 fs_manager.cc:359] Metadata directory not provided
I20260809 22:52:55.368243 22910 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:52:55.368573 22910 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:52:55.371467 22910 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315935617919-15814-0/minicluster-data/ts-2/data/instance:
uuid: "672643430e904bcfac288fd34367e30b"
format_stamp: "Formatted at 2026-08-09 22:52:55 on dist-test-slave-ncp9"
server_key: "4edc60044ffec3dab92af48ba52c5f43"
server_key_iv: "c06f72bb0612bd795fbb64117bf02867"
server_key_version: "encryptionkey@0"
I20260809 22:52:55.372258 22910 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315935617919-15814-0/minicluster-data/ts-2/wal/instance:
uuid: "672643430e904bcfac288fd34367e30b"
format_stamp: "Formatted at 2026-08-09 22:52:55 on dist-test-slave-ncp9"
server_key: "4edc60044ffec3dab92af48ba52c5f43"
server_key_iv: "c06f72bb0612bd795fbb64117bf02867"
server_key_version: "encryptionkey@0"
I20260809 22:52:55.376865 22910 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260809 22:52:55.379999 22934 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:52:55.381197 22910 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.003s
I20260809 22:52:55.381345 22910 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315935617919-15814-0/minicluster-data/ts-2/data,/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315935617919-15814-0/minicluster-data/ts-2/wal
uuid: "672643430e904bcfac288fd34367e30b"
format_stamp: "Formatted at 2026-08-09 22:52:55 on dist-test-slave-ncp9"
server_key: "4edc60044ffec3dab92af48ba52c5f43"
server_key_iv: "c06f72bb0612bd795fbb64117bf02867"
server_key_version: "encryptionkey@0"
I20260809 22:52:55.381531 22910 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315935617919-15814-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315935617919-15814-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315935617919-15814-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:52:55.395663 22910 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:52:55.399185 22910 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:52:55.399430 22910 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:52:55.400286 22910 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:52:55.401584 22910 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:52:55.401679 22910 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:52:55.401778 22910 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:52:55.401847 22910 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:52:55.441088 22910 rpc_server.cc:307] RPC server started. Bound to: 127.15.113.131:44693
I20260809 22:52:55.441138 23116 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.113.131:44693 every 8 connection(s)
I20260809 22:52:55.441900 22910 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315935617919-15814-0/minicluster-data/ts-2/data/info.pb
I20260809 22:52:55.444057 15814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskItYPJR/build/asan/bin/kudu as pid 22910
I20260809 22:52:55.444286 15814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315935617919-15814-0/minicluster-data/ts-2/wal/instance
I20260809 22:52:55.444830 15814 external_mini_cluster.cc:1428] Setting key 64f64a2e65d4e9f09300dea18f067569
Aug 09 22:52:55 dist-test-slave-ncp9 krb5kdc[22306](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315975, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.113.131@KRBTEST.COM for kudu/127.15.113.190@KRBTEST.COM
I20260809 22:52:55.462436 22697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:52:55.444675 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.113.131:36999 (local address 127.15.113.190:37153)
0809 22:52:55.445089 (+ 414us) server_negotiation.cc:206] Beginning negotiation
0809 22:52:55.445098 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 22:52:55.446434 (+ 1336us) server_negotiation.cc:407] Connection header received
0809 22:52:55.448926 (+ 2492us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:52:55.448933 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:52:55.449033 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:52:55.449113 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:52:55.450850 (+ 1737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:52:55.451788 (+ 938us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:52:55.453460 (+ 1672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:52:55.453886 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:52:55.453988 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:52:55.457033 (+ 3045us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:52:55.457038 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:52:55.457042 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:52:55.457072 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:52:55.459914 (+ 2842us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:52:55.460617 (+ 703us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:52:55.460621 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:52:55.460624 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:52:55.460744 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:52:55.461103 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:52:55.461106 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:52:55.461108 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:52:55.461380 (+ 272us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:52:55.461493 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0809 22:52:55.462026 (+ 533us) server_negotiation.cc:299] Negotiation successful
0809 22:52:55.462203 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260809 22:52:55.463263 23122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:52:55.445112 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:37153 (local address 127.15.113.131:36999)
0809 22:52:55.446192 (+ 1080us) negotiation.cc:107] Waiting for socket to connect
0809 22:52:55.446278 (+ 86us) client_negotiation.cc:174] Beginning negotiation
0809 22:52:55.448570 (+ 2292us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:52:55.449413 (+ 843us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:52:55.449425 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:52:55.450093 (+ 668us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:52:55.450686 (+ 593us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:52:55.450705 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:52:55.451935 (+ 1230us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:52:55.451940 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:52:55.453325 (+ 1385us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:52:55.453338 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:52:55.454055 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:52:55.454060 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:52:55.454229 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:52:55.454818 (+ 589us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:52:55.454843 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:52:55.456830 (+ 1987us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:52:55.460037 (+ 3207us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:52:55.460043 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:52:55.460053 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:52:55.460508 (+ 455us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:52:55.460822 (+ 314us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:52:55.460825 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:52:55.460827 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:52:55.461019 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:52:55.461469 (+ 450us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:52:55.461474 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:52:55.461746 (+ 272us) client_negotiation.cc:769] Sending connection context
0809 22:52:55.461957 (+ 211us) client_negotiation.cc:240] Negotiation successful
0809 22:52:55.462299 (+ 342us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":678,"thread_start_us":396,"threads_started":1}
I20260809 22:52:55.464527 23118 heartbeater.cc:347] Connected to a master server at 127.15.113.190:37153
I20260809 22:52:55.464777 23118 heartbeater.cc:464] Registering TS with master...
I20260809 22:52:55.465365 23118 heartbeater.cc:511] Master 127.15.113.190:37153 requested a full tablet report, sending...
I20260809 22:52:55.466699 22388 ts_manager.cc:194] Registered new tserver with Master: 672643430e904bcfac288fd34367e30b (127.15.113.131:44693)
I20260809 22:52:55.467444 22388 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.113.131@KRBTEST.COM'} at 127.15.113.131:36999
I20260809 22:52:55.472075 15814 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 22:52:55.487848 22697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:52:55.476511 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42040 (local address 127.15.113.190:37153)
0809 22:52:55.476664 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0809 22:52:55.476668 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 22:52:55.477220 (+ 552us) server_negotiation.cc:407] Connection header received
0809 22:52:55.477335 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:52:55.477338 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:52:55.477393 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:52:55.477466 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:52:55.478725 (+ 1259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:52:55.479419 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:52:55.480586 (+ 1167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:52:55.481066 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:52:55.481160 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:52:55.483196 (+ 2036us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:52:55.483200 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:52:55.483203 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:52:55.483227 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:52:55.485654 (+ 2427us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:52:55.486198 (+ 544us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:52:55.486202 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:52:55.486204 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:52:55.486322 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:52:55.486632 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:52:55.486637 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:52:55.486639 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:52:55.487013 (+ 374us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:52:55.487203 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0809 22:52:55.487483 (+ 280us) server_negotiation.cc:299] Negotiation successful
0809 22:52:55.487637 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260809 22:52:55.496287 22694 heartbeater.cc:503] Master 127.15.113.190:37153 was elected leader, sending a full tablet report...
I20260809 22:52:55.491284 22388 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42040:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 22:52:55.497932 22388 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 22:52:55.520358 23144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:52:55.509397 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46134 (local address 127.15.113.129:45677)
0809 22:52:55.511674 (+ 2277us) server_negotiation.cc:206] Beginning negotiation
0809 22:52:55.511696 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0809 22:52:55.511734 (+ 38us) server_negotiation.cc:407] Connection header received
0809 22:52:55.511857 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:52:55.511865 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:52:55.512152 (+ 287us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:52:55.512416 (+ 264us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:52:55.513202 (+ 786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:52:55.514899 (+ 1697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:52:55.517740 (+ 2841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:52:55.519202 (+ 1462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:52:55.519409 (+ 207us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:52:55.519594 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0809 22:52:55.519859 (+ 265us) server_negotiation.cc:299] Negotiation successful
0809 22:52:55.520023 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":955,"thread_start_us":1169,"threads_started":1}
I20260809 22:52:55.520380 23141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:52:55.509143 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.129:45677 (local address 127.0.0.1:46134)
0809 22:52:55.509938 (+ 795us) negotiation.cc:107] Waiting for socket to connect
0809 22:52:55.509973 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0809 22:52:55.510135 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:52:55.512447 (+ 2312us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:52:55.512456 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:52:55.512493 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:52:55.513003 (+ 510us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:52:55.513018 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:52:55.515031 (+ 2013us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:52:55.515035 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:52:55.517564 (+ 2529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:52:55.517582 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:52:55.519408 (+ 1826us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:52:55.519415 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:52:55.519604 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:52:55.519656 (+ 52us) client_negotiation.cc:769] Sending connection context
0809 22:52:55.519793 (+ 137us) client_negotiation.cc:240] Negotiation successful
0809 22:52:55.519983 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":563,"thread_start_us":375,"threads_started":1}
I20260809 22:52:55.521268 23147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:52:55.510784 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.130:40603 (local address 127.0.0.1:32954)
0809 22:52:55.513763 (+ 2979us) negotiation.cc:107] Waiting for socket to connect
0809 22:52:55.513778 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 22:52:55.513887 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:52:55.514440 (+ 553us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:52:55.514445 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:52:55.514468 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:52:55.514920 (+ 452us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:52:55.514934 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:52:55.516805 (+ 1871us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:52:55.516810 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:52:55.518529 (+ 1719us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:52:55.518541 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:52:55.520183 (+ 1642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:52:55.520189 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:52:55.520344 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:52:55.520387 (+ 43us) client_negotiation.cc:769] Sending connection context
0809 22:52:55.520877 (+ 490us) client_negotiation.cc:240] Negotiation successful
0809 22:52:55.521018 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2248,"mutex_wait_us":19,"thread_start_us":2447,"threads_started":1}
I20260809 22:52:55.521890 23148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:52:55.511173 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32954 (local address 127.15.113.130:40603)
0809 22:52:55.511774 (+ 601us) server_negotiation.cc:206] Beginning negotiation
0809 22:52:55.511784 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 22:52:55.513879 (+ 2095us) server_negotiation.cc:407] Connection header received
0809 22:52:55.514037 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:52:55.514043 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:52:55.514248 (+ 205us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:52:55.514440 (+ 192us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:52:55.515105 (+ 665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:52:55.516653 (+ 1548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:52:55.518681 (+ 2028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:52:55.519736 (+ 1055us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:52:55.519878 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:52:55.520039 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0809 22:52:55.521516 (+ 1477us) server_negotiation.cc:299] Negotiation successful
0809 22:52:55.521660 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":409,"thread_start_us":277,"threads_started":1}
I20260809 22:52:55.522637 23145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:52:55.510283 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.131:44693 (local address 127.0.0.1:54692)
0809 22:52:55.510790 (+ 507us) negotiation.cc:107] Waiting for socket to connect
0809 22:52:55.510806 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 22:52:55.510918 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:52:55.514493 (+ 3575us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:52:55.514499 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:52:55.514529 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:52:55.515299 (+ 770us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:52:55.515318 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:52:55.517241 (+ 1923us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:52:55.517246 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:52:55.519272 (+ 2026us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:52:55.519288 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:52:55.522027 (+ 2739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:52:55.522033 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:52:55.522203 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:52:55.522226 (+ 23us) client_negotiation.cc:769] Sending connection context
0809 22:52:55.522297 (+ 71us) client_negotiation.cc:240] Negotiation successful
0809 22:52:55.522378 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":378,"mutex_wait_us":3,"thread_start_us":668,"threads_started":1}
I20260809 22:52:55.522845 23146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:52:55.510778 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54692 (local address 127.15.113.131:44693)
0809 22:52:55.513911 (+ 3133us) server_negotiation.cc:206] Beginning negotiation
0809 22:52:55.513920 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 22:52:55.513953 (+ 33us) server_negotiation.cc:407] Connection header received
0809 22:52:55.514046 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:52:55.514053 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:52:55.514282 (+ 229us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:52:55.515143 (+ 861us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:52:55.515623 (+ 480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:52:55.517102 (+ 1479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:52:55.519449 (+ 2347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:52:55.521826 (+ 2377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:52:55.522022 (+ 196us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:52:55.522172 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0809 22:52:55.522352 (+ 180us) server_negotiation.cc:299] Negotiation successful
0809 22:52:55.522510 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2956,"thread_start_us":2782,"threads_started":1}
I20260809 22:52:55.524528 22586 tablet_service.cc:1467] Processing CreateTablet for tablet 0bf81e1cf4cf40f58610f16eced45a79 (DEFAULT_TABLE table=test-table [id=48f504480c8a44d2aaa5fd4854891cfb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:52:55.525981 22586 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0bf81e1cf4cf40f58610f16eced45a79. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:52:55.526031 23014 tablet_service.cc:1467] Processing CreateTablet for tablet 0bf81e1cf4cf40f58610f16eced45a79 (DEFAULT_TABLE table=test-table [id=48f504480c8a44d2aaa5fd4854891cfb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:52:55.527387 23014 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0bf81e1cf4cf40f58610f16eced45a79. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:52:55.528291 22804 tablet_service.cc:1467] Processing CreateTablet for tablet 0bf81e1cf4cf40f58610f16eced45a79 (DEFAULT_TABLE table=test-table [id=48f504480c8a44d2aaa5fd4854891cfb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:52:55.529579 22804 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0bf81e1cf4cf40f58610f16eced45a79. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:52:55.536934 23151 tablet_bootstrap.cc:492] T 0bf81e1cf4cf40f58610f16eced45a79 P 96a6a862316c48f3bdb1b24bc6c8037a: Bootstrap starting.
I20260809 22:52:55.540751 23151 tablet_bootstrap.cc:654] T 0bf81e1cf4cf40f58610f16eced45a79 P 96a6a862316c48f3bdb1b24bc6c8037a: Neither blocks nor log segments found. Creating new log.
I20260809 22:52:55.541504 23152 tablet_bootstrap.cc:492] T 0bf81e1cf4cf40f58610f16eced45a79 P 672643430e904bcfac288fd34367e30b: Bootstrap starting.
I20260809 22:52:55.542253 23151 log.cc:824] T 0bf81e1cf4cf40f58610f16eced45a79 P 96a6a862316c48f3bdb1b24bc6c8037a: Log is configured to *not* fsync() on all Append() calls
I20260809 22:52:55.543311 23153 tablet_bootstrap.cc:492] T 0bf81e1cf4cf40f58610f16eced45a79 P 758af62e22834fdfb9e8604c7f28a075: Bootstrap starting.
I20260809 22:52:55.544931 23152 tablet_bootstrap.cc:654] T 0bf81e1cf4cf40f58610f16eced45a79 P 672643430e904bcfac288fd34367e30b: Neither blocks nor log segments found. Creating new log.
I20260809 22:52:55.546151 23152 log.cc:824] T 0bf81e1cf4cf40f58610f16eced45a79 P 672643430e904bcfac288fd34367e30b: Log is configured to *not* fsync() on all Append() calls
I20260809 22:52:55.546298 23153 tablet_bootstrap.cc:654] T 0bf81e1cf4cf40f58610f16eced45a79 P 758af62e22834fdfb9e8604c7f28a075: Neither blocks nor log segments found. Creating new log.
I20260809 22:52:55.546643 23151 tablet_bootstrap.cc:492] T 0bf81e1cf4cf40f58610f16eced45a79 P 96a6a862316c48f3bdb1b24bc6c8037a: No bootstrap required, opened a new log
I20260809 22:52:55.546926 23151 ts_tablet_manager.cc:1397] T 0bf81e1cf4cf40f58610f16eced45a79 P 96a6a862316c48f3bdb1b24bc6c8037a: Time spent bootstrapping tablet: real 0.010s user 0.002s sys 0.006s
I20260809 22:52:55.547436 23153 log.cc:824] T 0bf81e1cf4cf40f58610f16eced45a79 P 758af62e22834fdfb9e8604c7f28a075: Log is configured to *not* fsync() on all Append() calls
I20260809 22:52:55.550021 23153 tablet_bootstrap.cc:492] T 0bf81e1cf4cf40f58610f16eced45a79 P 758af62e22834fdfb9e8604c7f28a075: No bootstrap required, opened a new log
I20260809 22:52:55.550246 23153 ts_tablet_manager.cc:1397] T 0bf81e1cf4cf40f58610f16eced45a79 P 758af62e22834fdfb9e8604c7f28a075: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260809 22:52:55.551357 23152 tablet_bootstrap.cc:492] T 0bf81e1cf4cf40f58610f16eced45a79 P 672643430e904bcfac288fd34367e30b: No bootstrap required, opened a new log
I20260809 22:52:55.551605 23152 ts_tablet_manager.cc:1397] T 0bf81e1cf4cf40f58610f16eced45a79 P 672643430e904bcfac288fd34367e30b: Time spent bootstrapping tablet: real 0.010s user 0.005s sys 0.002s
I20260809 22:52:55.556876 23151 raft_consensus.cc:348] T 0bf81e1cf4cf40f58610f16eced45a79 P 96a6a862316c48f3bdb1b24bc6c8037a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "96a6a862316c48f3bdb1b24bc6c8037a" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 45677 } } peers { permanent_uuid: "672643430e904bcfac288fd34367e30b" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 44693 } } peers { permanent_uuid: "758af62e22834fdfb9e8604c7f28a075" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 40603 } }
I20260809 22:52:55.557323 23151 raft_consensus.cc:374] T 0bf81e1cf4cf40f58610f16eced45a79 P 96a6a862316c48f3bdb1b24bc6c8037a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:52:55.557113 23153 raft_consensus.cc:348] T 0bf81e1cf4cf40f58610f16eced45a79 P 758af62e22834fdfb9e8604c7f28a075 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "96a6a862316c48f3bdb1b24bc6c8037a" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 45677 } } peers { permanent_uuid: "672643430e904bcfac288fd34367e30b" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 44693 } } peers { permanent_uuid: "758af62e22834fdfb9e8604c7f28a075" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 40603 } }
I20260809 22:52:55.557411 23151 raft_consensus.cc:729] T 0bf81e1cf4cf40f58610f16eced45a79 P 96a6a862316c48f3bdb1b24bc6c8037a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 96a6a862316c48f3bdb1b24bc6c8037a, State: Initialized, Role: FOLLOWER
I20260809 22:52:55.557510 23153 raft_consensus.cc:374] T 0bf81e1cf4cf40f58610f16eced45a79 P 758af62e22834fdfb9e8604c7f28a075 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:52:55.557610 23153 raft_consensus.cc:729] T 0bf81e1cf4cf40f58610f16eced45a79 P 758af62e22834fdfb9e8604c7f28a075 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 758af62e22834fdfb9e8604c7f28a075, State: Initialized, Role: FOLLOWER
I20260809 22:52:55.558117 23151 consensus_queue.cc:260] T 0bf81e1cf4cf40f58610f16eced45a79 P 96a6a862316c48f3bdb1b24bc6c8037a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "96a6a862316c48f3bdb1b24bc6c8037a" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 45677 } } peers { permanent_uuid: "672643430e904bcfac288fd34367e30b" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 44693 } } peers { permanent_uuid: "758af62e22834fdfb9e8604c7f28a075" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 40603 } }
I20260809 22:52:55.558269 23153 consensus_queue.cc:260] T 0bf81e1cf4cf40f58610f16eced45a79 P 758af62e22834fdfb9e8604c7f28a075 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "96a6a862316c48f3bdb1b24bc6c8037a" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 45677 } } peers { permanent_uuid: "672643430e904bcfac288fd34367e30b" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 44693 } } peers { permanent_uuid: "758af62e22834fdfb9e8604c7f28a075" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 40603 } }
I20260809 22:52:55.559713 22903 heartbeater.cc:503] Master 127.15.113.190:37153 was elected leader, sending a full tablet report...
I20260809 22:52:55.560227 23153 ts_tablet_manager.cc:1428] T 0bf81e1cf4cf40f58610f16eced45a79 P 758af62e22834fdfb9e8604c7f28a075: Time spent starting tablet: real 0.010s user 0.010s sys 0.000s
I20260809 22:52:55.561393 23151 ts_tablet_manager.cc:1428] T 0bf81e1cf4cf40f58610f16eced45a79 P 96a6a862316c48f3bdb1b24bc6c8037a: Time spent starting tablet: real 0.014s user 0.008s sys 0.004s
I20260809 22:52:55.562479 23152 raft_consensus.cc:348] T 0bf81e1cf4cf40f58610f16eced45a79 P 672643430e904bcfac288fd34367e30b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "96a6a862316c48f3bdb1b24bc6c8037a" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 45677 } } peers { permanent_uuid: "672643430e904bcfac288fd34367e30b" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 44693 } } peers { permanent_uuid: "758af62e22834fdfb9e8604c7f28a075" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 40603 } }
I20260809 22:52:55.562911 23152 raft_consensus.cc:374] T 0bf81e1cf4cf40f58610f16eced45a79 P 672643430e904bcfac288fd34367e30b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:52:55.563019 23152 raft_consensus.cc:729] T 0bf81e1cf4cf40f58610f16eced45a79 P 672643430e904bcfac288fd34367e30b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 672643430e904bcfac288fd34367e30b, State: Initialized, Role: FOLLOWER
I20260809 22:52:55.563598 23152 consensus_queue.cc:260] T 0bf81e1cf4cf40f58610f16eced45a79 P 672643430e904bcfac288fd34367e30b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "96a6a862316c48f3bdb1b24bc6c8037a" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 45677 } } peers { permanent_uuid: "672643430e904bcfac288fd34367e30b" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 44693 } } peers { permanent_uuid: "758af62e22834fdfb9e8604c7f28a075" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 40603 } }
I20260809 22:52:55.564981 23118 heartbeater.cc:503] Master 127.15.113.190:37153 was elected leader, sending a full tablet report...
I20260809 22:52:55.565608 23152 ts_tablet_manager.cc:1428] T 0bf81e1cf4cf40f58610f16eced45a79 P 672643430e904bcfac288fd34367e30b: Time spent starting tablet: real 0.014s user 0.006s sys 0.008s
W20260809 22:52:55.693352 23119 tablet.cc:2369] T 0bf81e1cf4cf40f58610f16eced45a79 P 672643430e904bcfac288fd34367e30b: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 22:52:55.710254 22906 tablet.cc:2369] T 0bf81e1cf4cf40f58610f16eced45a79 P 758af62e22834fdfb9e8604c7f28a075: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 22:52:55.719030 22695 tablet.cc:2369] T 0bf81e1cf4cf40f58610f16eced45a79 P 96a6a862316c48f3bdb1b24bc6c8037a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 22:52:55.742837 23160 raft_consensus.cc:482] T 0bf81e1cf4cf40f58610f16eced45a79 P 758af62e22834fdfb9e8604c7f28a075 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 22:52:55.743162 23160 raft_consensus.cc:504] T 0bf81e1cf4cf40f58610f16eced45a79 P 758af62e22834fdfb9e8604c7f28a075 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "96a6a862316c48f3bdb1b24bc6c8037a" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 45677 } } peers { permanent_uuid: "672643430e904bcfac288fd34367e30b" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 44693 } } peers { permanent_uuid: "758af62e22834fdfb9e8604c7f28a075" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 40603 } }
I20260809 22:52:55.745324 23160 leader_election.cc:288] T 0bf81e1cf4cf40f58610f16eced45a79 P 758af62e22834fdfb9e8604c7f28a075 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 96a6a862316c48f3bdb1b24bc6c8037a (127.15.113.129:45677), 672643430e904bcfac288fd34367e30b (127.15.113.131:44693)
I20260809 22:52:55.756338 23144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:52:55.746446 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.113.130:49243 (local address 127.15.113.129:45677)
0809 22:52:55.746628 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0809 22:52:55.746635 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:52:55.750704 (+ 4069us) server_negotiation.cc:407] Connection header received
0809 22:52:55.750963 (+ 259us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:52:55.750969 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:52:55.751056 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:52:55.751129 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:52:55.752072 (+ 943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:52:55.752810 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:52:55.754567 (+ 1757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:52:55.755315 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:52:55.755461 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:52:55.755536 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0809 22:52:55.756046 (+ 510us) server_negotiation.cc:299] Negotiation successful
0809 22:52:55.756149 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260809 22:52:55.756510 23163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:52:55.745908 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.129:45677 (local address 127.15.113.130:49243)
0809 22:52:55.750581 (+ 4673us) negotiation.cc:107] Waiting for socket to connect
0809 22:52:55.750601 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 22:52:55.750768 (+ 167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:52:55.751374 (+ 606us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:52:55.751380 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:52:55.751412 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:52:55.751925 (+ 513us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:52:55.751940 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:52:55.752963 (+ 1023us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:52:55.752969 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:52:55.754400 (+ 1431us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:52:55.754416 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:52:55.755689 (+ 1273us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:52:55.755695 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:52:55.755868 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:52:55.755902 (+ 34us) client_negotiation.cc:769] Sending connection context
0809 22:52:55.755993 (+ 91us) client_negotiation.cc:240] Negotiation successful
0809 22:52:55.756116 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4541,"mutex_wait_us":3,"thread_start_us":4383,"threads_started":1}
I20260809 22:52:55.757416 22620 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0bf81e1cf4cf40f58610f16eced45a79" candidate_uuid: "758af62e22834fdfb9e8604c7f28a075" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96a6a862316c48f3bdb1b24bc6c8037a" is_pre_election: true
I20260809 22:52:55.757658 23146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:52:55.750976 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.113.130:51619 (local address 127.15.113.131:44693)
0809 22:52:55.751174 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0809 22:52:55.751180 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:52:55.752160 (+ 980us) server_negotiation.cc:407] Connection header received
0809 22:52:55.752322 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:52:55.752328 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:52:55.752393 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:52:55.752465 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:52:55.753026 (+ 561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:52:55.753728 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:52:55.755443 (+ 1715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:52:55.756636 (+ 1193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:52:55.756789 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:52:55.756860 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0809 22:52:55.757306 (+ 446us) server_negotiation.cc:299] Negotiation successful
0809 22:52:55.757431 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260809 22:52:55.757697 23164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:52:55.750947 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.131:44693 (local address 127.15.113.130:51619)
0809 22:52:55.752052 (+ 1105us) negotiation.cc:107] Waiting for socket to connect
0809 22:52:55.752067 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 22:52:55.752195 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:52:55.752503 (+ 308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:52:55.752507 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:52:55.752531 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:52:55.752849 (+ 318us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:52:55.752858 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:52:55.753858 (+ 1000us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:52:55.753864 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:52:55.755276 (+ 1412us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:52:55.755292 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:52:55.757031 (+ 1739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:52:55.757036 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:52:55.757158 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:52:55.757181 (+ 23us) client_negotiation.cc:769] Sending connection context
0809 22:52:55.757260 (+ 79us) client_negotiation.cc:240] Negotiation successful
0809 22:52:55.757372 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":997,"thread_start_us":779,"threads_started":1}
I20260809 22:52:55.757937 22620 raft_consensus.cc:2454] T 0bf81e1cf4cf40f58610f16eced45a79 P 96a6a862316c48f3bdb1b24bc6c8037a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 758af62e22834fdfb9e8604c7f28a075 in term 0.
I20260809 22:52:55.758610 22734 leader_election.cc:302] T 0bf81e1cf4cf40f58610f16eced45a79 P 758af62e22834fdfb9e8604c7f28a075 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 758af62e22834fdfb9e8604c7f28a075, 96a6a862316c48f3bdb1b24bc6c8037a; no voters:
I20260809 22:52:55.758595 23046 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0bf81e1cf4cf40f58610f16eced45a79" candidate_uuid: "758af62e22834fdfb9e8604c7f28a075" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "672643430e904bcfac288fd34367e30b" is_pre_election: true
I20260809 22:52:55.758971 23160 raft_consensus.cc:2781] T 0bf81e1cf4cf40f58610f16eced45a79 P 758af62e22834fdfb9e8604c7f28a075 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 22:52:55.759102 23160 raft_consensus.cc:482] T 0bf81e1cf4cf40f58610f16eced45a79 P 758af62e22834fdfb9e8604c7f28a075 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 22:52:55.759043 23046 raft_consensus.cc:2454] T 0bf81e1cf4cf40f58610f16eced45a79 P 672643430e904bcfac288fd34367e30b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 758af62e22834fdfb9e8604c7f28a075 in term 0.
I20260809 22:52:55.759199 23160 raft_consensus.cc:3037] T 0bf81e1cf4cf40f58610f16eced45a79 P 758af62e22834fdfb9e8604c7f28a075 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:52:55.761461 23160 raft_consensus.cc:504] T 0bf81e1cf4cf40f58610f16eced45a79 P 758af62e22834fdfb9e8604c7f28a075 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "96a6a862316c48f3bdb1b24bc6c8037a" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 45677 } } peers { permanent_uuid: "672643430e904bcfac288fd34367e30b" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 44693 } } peers { permanent_uuid: "758af62e22834fdfb9e8604c7f28a075" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 40603 } }
I20260809 22:52:55.761988 23160 leader_election.cc:288] T 0bf81e1cf4cf40f58610f16eced45a79 P 758af62e22834fdfb9e8604c7f28a075 [CANDIDATE]: Term 1 election: Requested vote from peers 96a6a862316c48f3bdb1b24bc6c8037a (127.15.113.129:45677), 672643430e904bcfac288fd34367e30b (127.15.113.131:44693)
I20260809 22:52:55.762476 22620 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0bf81e1cf4cf40f58610f16eced45a79" candidate_uuid: "758af62e22834fdfb9e8604c7f28a075" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96a6a862316c48f3bdb1b24bc6c8037a"
I20260809 22:52:55.762544 23046 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0bf81e1cf4cf40f58610f16eced45a79" candidate_uuid: "758af62e22834fdfb9e8604c7f28a075" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "672643430e904bcfac288fd34367e30b"
I20260809 22:52:55.762686 22620 raft_consensus.cc:3037] T 0bf81e1cf4cf40f58610f16eced45a79 P 96a6a862316c48f3bdb1b24bc6c8037a [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:52:55.762712 23046 raft_consensus.cc:3037] T 0bf81e1cf4cf40f58610f16eced45a79 P 672643430e904bcfac288fd34367e30b [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:52:55.765774 23046 raft_consensus.cc:2454] T 0bf81e1cf4cf40f58610f16eced45a79 P 672643430e904bcfac288fd34367e30b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 758af62e22834fdfb9e8604c7f28a075 in term 1.
I20260809 22:52:55.765774 22620 raft_consensus.cc:2454] T 0bf81e1cf4cf40f58610f16eced45a79 P 96a6a862316c48f3bdb1b24bc6c8037a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 758af62e22834fdfb9e8604c7f28a075 in term 1.
I20260809 22:52:55.766134 22734 leader_election.cc:302] T 0bf81e1cf4cf40f58610f16eced45a79 P 758af62e22834fdfb9e8604c7f28a075 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 758af62e22834fdfb9e8604c7f28a075, 96a6a862316c48f3bdb1b24bc6c8037a; no voters:
I20260809 22:52:55.766412 23160 raft_consensus.cc:2781] T 0bf81e1cf4cf40f58610f16eced45a79 P 758af62e22834fdfb9e8604c7f28a075 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 22:52:55.766819 23160 raft_consensus.cc:686] T 0bf81e1cf4cf40f58610f16eced45a79 P 758af62e22834fdfb9e8604c7f28a075 [term 1 LEADER]: Becoming Leader. State: Replica: 758af62e22834fdfb9e8604c7f28a075, State: Running, Role: LEADER
I20260809 22:52:55.767299 23160 consensus_queue.cc:237] T 0bf81e1cf4cf40f58610f16eced45a79 P 758af62e22834fdfb9e8604c7f28a075 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "96a6a862316c48f3bdb1b24bc6c8037a" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 45677 } } peers { permanent_uuid: "672643430e904bcfac288fd34367e30b" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 44693 } } peers { permanent_uuid: "758af62e22834fdfb9e8604c7f28a075" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 40603 } }
I20260809 22:52:55.772008 22387 catalog_manager.cc:5495] T 0bf81e1cf4cf40f58610f16eced45a79 P 758af62e22834fdfb9e8604c7f28a075 reported cstate change: term changed from 0 to 1, leader changed from <none> to 758af62e22834fdfb9e8604c7f28a075 (127.15.113.130). New cstate: current_term: 1 leader_uuid: "758af62e22834fdfb9e8604c7f28a075" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "96a6a862316c48f3bdb1b24bc6c8037a" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 45677 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "672643430e904bcfac288fd34367e30b" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 44693 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "758af62e22834fdfb9e8604c7f28a075" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 40603 } health_report { overall_health: HEALTHY } } }
I20260809 22:52:55.809891 23148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:52:55.805109 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32964 (local address 127.15.113.130:40603)
0809 22:52:55.805290 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0809 22:52:55.805295 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:52:55.805458 (+ 163us) server_negotiation.cc:407] Connection header received
0809 22:52:55.805624 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:52:55.805628 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:52:55.805712 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:52:55.805831 (+ 119us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 22:52:55.806114 (+ 283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:52:55.806817 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:52:55.808056 (+ 1239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:52:55.808447 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:52:55.808529 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:52:55.808926 (+ 397us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 22:52:55.809458 (+ 532us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 22:52:55.809562 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0809 22:52:55.809624 (+ 62us) server_negotiation.cc:299] Negotiation successful
0809 22:52:55.809685 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260809 22:52:55.817554 23046 raft_consensus.cc:1261] T 0bf81e1cf4cf40f58610f16eced45a79 P 672643430e904bcfac288fd34367e30b [term 1 FOLLOWER]: Refusing update from remote peer 758af62e22834fdfb9e8604c7f28a075: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 22:52:55.817592 22620 raft_consensus.cc:1261] T 0bf81e1cf4cf40f58610f16eced45a79 P 96a6a862316c48f3bdb1b24bc6c8037a [term 1 FOLLOWER]: Refusing update from remote peer 758af62e22834fdfb9e8604c7f28a075: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 22:52:55.818428 23165 consensus_queue.cc:1035] T 0bf81e1cf4cf40f58610f16eced45a79 P 758af62e22834fdfb9e8604c7f28a075 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a6a862316c48f3bdb1b24bc6c8037a" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 45677 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 22:52:55.818810 23160 consensus_queue.cc:1035] T 0bf81e1cf4cf40f58610f16eced45a79 P 758af62e22834fdfb9e8604c7f28a075 [LEADER]: Connected to new peer: Peer: permanent_uuid: "672643430e904bcfac288fd34367e30b" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 44693 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 22:52:55.843159 22388 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42040:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 22:52:55.843384 22388 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42040:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 22:52:55.846803 22388 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d9d8c645b1514b3ebab412b28bae9cfa: Sending DeleteTablet for 3 replicas of tablet 0bf81e1cf4cf40f58610f16eced45a79
I20260809 22:52:55.847997 22586 tablet_service.cc:1514] Processing DeleteTablet for tablet 0bf81e1cf4cf40f58610f16eced45a79 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:52:55 UTC) from {username='kudu', principal='kudu/127.15.113.190@KRBTEST.COM'} at 127.0.0.1:46134
I20260809 22:52:55.848263 23014 tablet_service.cc:1514] Processing DeleteTablet for tablet 0bf81e1cf4cf40f58610f16eced45a79 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:52:55 UTC) from {username='kudu', principal='kudu/127.15.113.190@KRBTEST.COM'} at 127.0.0.1:54692
I20260809 22:52:55.848745 22804 tablet_service.cc:1514] Processing DeleteTablet for tablet 0bf81e1cf4cf40f58610f16eced45a79 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:52:55 UTC) from {username='kudu', principal='kudu/127.15.113.190@KRBTEST.COM'} at 127.0.0.1:32954
I20260809 22:52:55.848937 23188 tablet_replica.cc:330] T 0bf81e1cf4cf40f58610f16eced45a79 P 96a6a862316c48f3bdb1b24bc6c8037a: stopping tablet replica
I20260809 22:52:55.849341 23188 raft_consensus.cc:2229] T 0bf81e1cf4cf40f58610f16eced45a79 P 96a6a862316c48f3bdb1b24bc6c8037a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 22:52:55.849480 15814 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskItYPJR/build/asan/bin/kudu pid 22479
I20260809 22:52:55.849833 23188 raft_consensus.cc:2258] T 0bf81e1cf4cf40f58610f16eced45a79 P 96a6a862316c48f3bdb1b24bc6c8037a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 22:52:55.852108 23188 ts_tablet_manager.cc:1905] T 0bf81e1cf4cf40f58610f16eced45a79 P 96a6a862316c48f3bdb1b24bc6c8037a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 22:52:55.858289 23191 tablet_replica.cc:330] T 0bf81e1cf4cf40f58610f16eced45a79 P 758af62e22834fdfb9e8604c7f28a075: stopping tablet replica
I20260809 22:52:55.859141 23188 ts_tablet_manager.cc:1918] T 0bf81e1cf4cf40f58610f16eced45a79 P 96a6a862316c48f3bdb1b24bc6c8037a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 22:52:55.859313 23188 log.cc:1196] T 0bf81e1cf4cf40f58610f16eced45a79 P 96a6a862316c48f3bdb1b24bc6c8037a: Deleting WAL directory at /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315935617919-15814-0/minicluster-data/ts-0/wal/wals/0bf81e1cf4cf40f58610f16eced45a79
I20260809 22:52:55.859758 23191 raft_consensus.cc:2229] T 0bf81e1cf4cf40f58610f16eced45a79 P 758af62e22834fdfb9e8604c7f28a075 [term 1 LEADER]: Raft consensus shutting down.
I20260809 22:52:55.859861 23188 ts_tablet_manager.cc:1939] T 0bf81e1cf4cf40f58610f16eced45a79 P 96a6a862316c48f3bdb1b24bc6c8037a: Deleting consensus metadata
I20260809 22:52:55.860739 23191 raft_consensus.cc:2258] T 0bf81e1cf4cf40f58610f16eced45a79 P 758af62e22834fdfb9e8604c7f28a075 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 22:52:55.861651 22363 catalog_manager.cc:4841] TS 96a6a862316c48f3bdb1b24bc6c8037a (127.15.113.129:45677): tablet 0bf81e1cf4cf40f58610f16eced45a79 (table test-table [id=48f504480c8a44d2aaa5fd4854891cfb]) successfully deleted
I20260809 22:52:55.863521 23191 ts_tablet_manager.cc:1905] T 0bf81e1cf4cf40f58610f16eced45a79 P 758af62e22834fdfb9e8604c7f28a075: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 22:52:55.865183 23189 tablet_replica.cc:330] T 0bf81e1cf4cf40f58610f16eced45a79 P 672643430e904bcfac288fd34367e30b: stopping tablet replica
I20260809 22:52:55.865566 23189 raft_consensus.cc:2229] T 0bf81e1cf4cf40f58610f16eced45a79 P 672643430e904bcfac288fd34367e30b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 22:52:55.866005 23189 raft_consensus.cc:2258] T 0bf81e1cf4cf40f58610f16eced45a79 P 672643430e904bcfac288fd34367e30b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 22:52:55.868204 23189 ts_tablet_manager.cc:1905] T 0bf81e1cf4cf40f58610f16eced45a79 P 672643430e904bcfac288fd34367e30b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 22:52:55.870865 23191 ts_tablet_manager.cc:1918] T 0bf81e1cf4cf40f58610f16eced45a79 P 758af62e22834fdfb9e8604c7f28a075: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 22:52:55.871026 23191 log.cc:1196] T 0bf81e1cf4cf40f58610f16eced45a79 P 758af62e22834fdfb9e8604c7f28a075: Deleting WAL directory at /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315935617919-15814-0/minicluster-data/ts-1/wal/wals/0bf81e1cf4cf40f58610f16eced45a79
I20260809 22:52:55.871618 23191 ts_tablet_manager.cc:1939] T 0bf81e1cf4cf40f58610f16eced45a79 P 758af62e22834fdfb9e8604c7f28a075: Deleting consensus metadata
Aug 09 22:52:55 dist-test-slave-ncp9 krb5kdc[22306](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315973, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.113.129@KRBTEST.COM
I20260809 22:52:55.873329 22362 catalog_manager.cc:4841] TS 758af62e22834fdfb9e8604c7f28a075 (127.15.113.130:40603): tablet 0bf81e1cf4cf40f58610f16eced45a79 (table test-table [id=48f504480c8a44d2aaa5fd4854891cfb]) successfully deleted
I20260809 22:52:55.874836 23189 ts_tablet_manager.cc:1918] T 0bf81e1cf4cf40f58610f16eced45a79 P 672643430e904bcfac288fd34367e30b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 22:52:55.874979 23189 log.cc:1196] T 0bf81e1cf4cf40f58610f16eced45a79 P 672643430e904bcfac288fd34367e30b: Deleting WAL directory at /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315935617919-15814-0/minicluster-data/ts-2/wal/wals/0bf81e1cf4cf40f58610f16eced45a79
I20260809 22:52:55.875494 23189 ts_tablet_manager.cc:1939] T 0bf81e1cf4cf40f58610f16eced45a79 P 672643430e904bcfac288fd34367e30b: Deleting consensus metadata
I20260809 22:52:55.876857 22363 catalog_manager.cc:4841] TS 672643430e904bcfac288fd34367e30b (127.15.113.131:44693): tablet 0bf81e1cf4cf40f58610f16eced45a79 (table test-table [id=48f504480c8a44d2aaa5fd4854891cfb]) successfully deleted
I20260809 22:52:55.880910 23144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:52:55.850738 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46148 (local address 127.15.113.129:45677)
0809 22:52:55.850982 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0809 22:52:55.850989 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:52:55.862909 (+ 11920us) server_negotiation.cc:407] Connection header received
0809 22:52:55.863320 (+ 411us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:52:55.863326 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:52:55.863419 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:52:55.863490 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:52:55.865396 (+ 1906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:52:55.866440 (+ 1044us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:52:55.868259 (+ 1819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:52:55.868837 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:52:55.868971 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:52:55.874341 (+ 5370us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:52:55.874357 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:52:55.874375 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:52:55.874415 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:52:55.878198 (+ 3783us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:52:55.878814 (+ 616us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:52:55.878824 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:52:55.878827 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:52:55.879001 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:52:55.879317 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:52:55.879322 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:52:55.879326 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:52:55.879781 (+ 455us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:52:55.880102 (+ 321us) server_negotiation.cc:1035] Waiting for connection context
0809 22:52:55.880267 (+ 165us) server_negotiation.cc:299] Negotiation successful
0809 22:52:55.880496 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":103}
I20260809 22:52:55.881517 22685 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:52:56.017710 15814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskItYPJR/build/asan/bin/kudu with pid 22479
I20260809 22:52:56.040011 15814 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskItYPJR/build/asan/bin/kudu pid 22703
Aug 09 22:52:56 dist-test-slave-ncp9 krb5kdc[22306](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315973, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.113.130@KRBTEST.COM
I20260809 22:52:56.056797 23148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:52:56.040736 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32978 (local address 127.15.113.130:40603)
0809 22:52:56.040953 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0809 22:52:56.040960 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:52:56.040992 (+ 32us) server_negotiation.cc:407] Connection header received
0809 22:52:56.041144 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:52:56.041150 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:52:56.041242 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:52:56.041354 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:52:56.042777 (+ 1423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:52:56.044019 (+ 1242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:52:56.045441 (+ 1422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:52:56.046124 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:52:56.046248 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:52:56.050335 (+ 4087us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:52:56.050350 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:52:56.050370 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:52:56.050424 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:52:56.054184 (+ 3760us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:52:56.054827 (+ 643us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:52:56.054837 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:52:56.054841 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:52:56.055014 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:52:56.055309 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:52:56.055313 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:52:56.055317 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:52:56.055732 (+ 415us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:52:56.056042 (+ 310us) server_negotiation.cc:1035] Waiting for connection context
0809 22:52:56.056208 (+ 166us) server_negotiation.cc:299] Negotiation successful
0809 22:52:56.056414 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260809 22:52:56.057613 22895 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:52:56.169796 15814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskItYPJR/build/asan/bin/kudu with pid 22703
I20260809 22:52:56.192734 15814 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskItYPJR/build/asan/bin/kudu pid 22910
Aug 09 22:52:56 dist-test-slave-ncp9 krb5kdc[22306](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315973, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.113.131@KRBTEST.COM
I20260809 22:52:56.207005 23146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:52:56.193464 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54698 (local address 127.15.113.131:44693)
0809 22:52:56.193618 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0809 22:52:56.193630 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 22:52:56.193668 (+ 38us) server_negotiation.cc:407] Connection header received
0809 22:52:56.193823 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:52:56.193827 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:52:56.193892 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:52:56.193985 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:52:56.195331 (+ 1346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:52:56.196059 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:52:56.197388 (+ 1329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:52:56.197765 (+ 377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:52:56.197855 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:52:56.201790 (+ 3935us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:52:56.201805 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:52:56.201823 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:52:56.201856 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:52:56.204368 (+ 2512us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:52:56.205030 (+ 662us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:52:56.205038 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:52:56.205040 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:52:56.205170 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:52:56.205536 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:52:56.205540 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:52:56.205542 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:52:56.205907 (+ 365us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:52:56.206192 (+ 285us) server_negotiation.cc:1035] Waiting for connection context
0809 22:52:56.206495 (+ 303us) server_negotiation.cc:299] Negotiation successful
0809 22:52:56.206722 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260809 22:52:56.207612 23111 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:52:56.293327 15814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskItYPJR/build/asan/bin/kudu with pid 22910
I20260809 22:52:56.313578 15814 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskItYPJR/build/asan/bin/kudu pid 22330
I20260809 22:52:56.314690 22431 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:52:56.426280 15814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskItYPJR/build/asan/bin/kudu with pid 22330
2026-08-09T22:52:56Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (4334 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-08-09T22:52:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T22:52:56Z Disabled control of system clock
I20260809 22:52:56.507203 15814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskItYPJR/build/asan/bin/kudu
/tmp/dist-test-taskItYPJR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786315935617919-15814-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786315935617919-15814-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786315935617919-15814-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786315935617919-15814-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.113.190:42223
--webserver_interface=127.15.113.190
--webserver_port=0
--builtin_ntp_servers=127.15.113.148:38805
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.113.190:42223
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260809 22:52:56.832267 23203 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:52:56.832542 23203 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:52:56.832633 23203 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:52:56.839730 23203 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 22:52:56.839826 23203 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:52:56.839877 23203 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:52:56.839931 23203 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 22:52:56.839980 23203 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 22:52:56.848799 23203 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.113.148:38805
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786315935617919-15814-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786315935617919-15814-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.113.190:42223
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.113.190:42223
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786315935617919-15814-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.113.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786315935617919-15814-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:52:56.850014 23203 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:52:56.851713 23203 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:52:56.862717 23213 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:52:56.862824 23211 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:52:56.862812 23216 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:52:56.863746 23203 server_base.cc:1034] running on GCE node
I20260809 22:52:56.864465 23203 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:52:56.865814 23203 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:52:56.867014 23203 hybrid_clock.cc:648] HybridClock initialized: now 1786315976866967 us; error 34 us; skew 500 ppm
I20260809 22:52:56.867374 23203 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 22:52:56.869531 23203 webserver.cc:460] Webserver started at http://127.15.113.190:44475/ using document root <none> and password file <none>
I20260809 22:52:56.870199 23203 fs_manager.cc:359] Metadata directory not provided
I20260809 22:52:56.870292 23203 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:52:56.870605 23203 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:52:56.873484 23203 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786315935617919-15814-0/minicluster-data/master-0/data/instance:
uuid: "71056d810a3c42a99a5382ad97d48c18"
format_stamp: "Formatted at 2026-08-09 22:52:56 on dist-test-slave-ncp9"
server_key: "6c8b3bfaa397a417b1bea03e4c5127c0"
server_key_iv: "1faf9b340f732d92ec957e58862b0572"
server_key_version: "encryptionkey@0"
I20260809 22:52:56.874287 23203 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786315935617919-15814-0/minicluster-data/master-0/wal/instance:
uuid: "71056d810a3c42a99a5382ad97d48c18"
format_stamp: "Formatted at 2026-08-09 22:52:56 on dist-test-slave-ncp9"
server_key: "6c8b3bfaa397a417b1bea03e4c5127c0"
server_key_iv: "1faf9b340f732d92ec957e58862b0572"
server_key_version: "encryptionkey@0"
I20260809 22:52:56.879194 23203 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.003s
I20260809 22:52:56.882437 23227 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:52:56.883669 23203 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260809 22:52:56.883819 23203 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786315935617919-15814-0/minicluster-data/master-0/data,/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786315935617919-15814-0/minicluster-data/master-0/wal
uuid: "71056d810a3c42a99a5382ad97d48c18"
format_stamp: "Formatted at 2026-08-09 22:52:56 on dist-test-slave-ncp9"
server_key: "6c8b3bfaa397a417b1bea03e4c5127c0"
server_key_iv: "1faf9b340f732d92ec957e58862b0572"
server_key_version: "encryptionkey@0"
I20260809 22:52:56.884012 23203 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786315935617919-15814-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786315935617919-15814-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786315935617919-15814-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:52:56.889550 23203 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:52:56.890295 23203 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:52:56.890568 23203 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:52:56.907389 23203 rpc_server.cc:307] RPC server started. Bound to: 127.15.113.190:42223
I20260809 22:52:56.907421 23314 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.113.190:42223 every 8 connection(s)
I20260809 22:52:56.908617 23203 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786315935617919-15814-0/minicluster-data/master-0/data/info.pb
I20260809 22:52:56.912204 23315 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:52:56.917182 15814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskItYPJR/build/asan/bin/kudu as pid 23203
I20260809 22:52:56.917343 15814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786315935617919-15814-0/minicluster-data/master-0/wal/instance
I20260809 22:52:56.917964 15814 external_mini_cluster.cc:1428] Setting key 46a111d089bd8e3d9b948a14667b0dea
I20260809 22:52:56.922693 23315 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 71056d810a3c42a99a5382ad97d48c18: Bootstrap starting.
I20260809 22:52:56.926010 23315 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 71056d810a3c42a99a5382ad97d48c18: Neither blocks nor log segments found. Creating new log.
I20260809 22:52:56.926077 23319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:52:56.919974 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55786 (local address 127.15.113.190:42223)
0809 22:52:56.921024 (+ 1050us) server_negotiation.cc:206] Beginning negotiation
0809 22:52:56.921035 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 22:52:56.921080 (+ 45us) server_negotiation.cc:407] Connection header received
0809 22:52:56.922670 (+ 1590us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:52:56.922695 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:52:56.923097 (+ 402us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:52:56.923440 (+ 343us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:52:56.924516 (+ 1076us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:52:56.924523 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:52:56.924535 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 22:52:56.924549 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:52:56.924611 (+ 62us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 22:52:56.924639 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:52:56.924843 (+ 204us) server_negotiation.cc:1035] Waiting for connection context
0809 22:52:56.925012 (+ 169us) server_negotiation.cc:299] Negotiation successful
0809 22:52:56.925246 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":593,"thread_start_us":351,"threads_started":1}
I20260809 22:52:56.927310 23315 log.cc:824] T 00000000000000000000000000000000 P 71056d810a3c42a99a5382ad97d48c18: Log is configured to *not* fsync() on all Append() calls
I20260809 22:52:56.930223 23315 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 71056d810a3c42a99a5382ad97d48c18: No bootstrap required, opened a new log
I20260809 22:52:56.937704 23315 raft_consensus.cc:348] T 00000000000000000000000000000000 P 71056d810a3c42a99a5382ad97d48c18 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "71056d810a3c42a99a5382ad97d48c18" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 42223 } }
I20260809 22:52:56.938090 23315 raft_consensus.cc:374] T 00000000000000000000000000000000 P 71056d810a3c42a99a5382ad97d48c18 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:52:56.938192 23315 raft_consensus.cc:729] T 00000000000000000000000000000000 P 71056d810a3c42a99a5382ad97d48c18 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 71056d810a3c42a99a5382ad97d48c18, State: Initialized, Role: FOLLOWER
I20260809 22:52:56.938747 23315 consensus_queue.cc:260] T 00000000000000000000000000000000 P 71056d810a3c42a99a5382ad97d48c18 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "71056d810a3c42a99a5382ad97d48c18" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 42223 } }
I20260809 22:52:56.939005 23315 raft_consensus.cc:388] T 00000000000000000000000000000000 P 71056d810a3c42a99a5382ad97d48c18 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 22:52:56.939141 23315 raft_consensus.cc:482] T 00000000000000000000000000000000 P 71056d810a3c42a99a5382ad97d48c18 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 22:52:56.939311 23315 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 71056d810a3c42a99a5382ad97d48c18 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:52:56.941633 23315 raft_consensus.cc:504] T 00000000000000000000000000000000 P 71056d810a3c42a99a5382ad97d48c18 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "71056d810a3c42a99a5382ad97d48c18" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 42223 } }
I20260809 22:52:56.942233 23315 leader_election.cc:302] T 00000000000000000000000000000000 P 71056d810a3c42a99a5382ad97d48c18 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 71056d810a3c42a99a5382ad97d48c18; no voters:
I20260809 22:52:56.942911 23315 leader_election.cc:288] T 00000000000000000000000000000000 P 71056d810a3c42a99a5382ad97d48c18 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 22:52:56.942998 23322 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 71056d810a3c42a99a5382ad97d48c18 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 22:52:56.943738 23322 raft_consensus.cc:686] T 00000000000000000000000000000000 P 71056d810a3c42a99a5382ad97d48c18 [term 1 LEADER]: Becoming Leader. State: Replica: 71056d810a3c42a99a5382ad97d48c18, State: Running, Role: LEADER
I20260809 22:52:56.944284 23322 consensus_queue.cc:237] T 00000000000000000000000000000000 P 71056d810a3c42a99a5382ad97d48c18 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "71056d810a3c42a99a5382ad97d48c18" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 42223 } }
I20260809 22:52:56.945065 23315 sys_catalog.cc:564] T 00000000000000000000000000000000 P 71056d810a3c42a99a5382ad97d48c18 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 22:52:56.947057 23324 sys_catalog.cc:455] T 00000000000000000000000000000000 P 71056d810a3c42a99a5382ad97d48c18 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "71056d810a3c42a99a5382ad97d48c18" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "71056d810a3c42a99a5382ad97d48c18" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 42223 } } }
I20260809 22:52:56.947530 23324 sys_catalog.cc:458] T 00000000000000000000000000000000 P 71056d810a3c42a99a5382ad97d48c18 [sys.catalog]: This master's current role is: LEADER
I20260809 22:52:56.948961 23335 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 22:52:56.947371 23325 sys_catalog.cc:455] T 00000000000000000000000000000000 P 71056d810a3c42a99a5382ad97d48c18 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 71056d810a3c42a99a5382ad97d48c18. Latest consensus state: current_term: 1 leader_uuid: "71056d810a3c42a99a5382ad97d48c18" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "71056d810a3c42a99a5382ad97d48c18" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 42223 } } }
I20260809 22:52:56.949493 23325 sys_catalog.cc:458] T 00000000000000000000000000000000 P 71056d810a3c42a99a5382ad97d48c18 [sys.catalog]: This master's current role is: LEADER
I20260809 22:52:56.953789 23335 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 22:52:56.962477 23335 catalog_manager.cc:1347] Generated new cluster ID: ef8bc7d50f6d499bad7d38f769cc9c33
I20260809 22:52:56.962664 23335 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 22:52:57.008618 23335 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 22:52:57.009232 23335 catalog_manager.cc:1504] Loading token signing keys...
I20260809 22:52:57.018420 23335 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 71056d810a3c42a99a5382ad97d48c18: Generated new TSK 0
I20260809 22:52:57.019209 23335 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 22:52:57.026556 15814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskItYPJR/build/asan/bin/kudu
/tmp/dist-test-taskItYPJR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786315935617919-15814-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786315935617919-15814-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786315935617919-15814-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786315935617919-15814-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.113.129:0
--local_ip_for_outbound_sockets=127.15.113.129
--webserver_interface=127.15.113.129
--webserver_port=0
--tserver_master_addrs=127.15.113.190:42223
--builtin_ntp_servers=127.15.113.148:38805
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260809 22:52:57.365108 23352 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:52:57.365355 23352 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:52:57.365479 23352 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:52:57.372422 23352 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:52:57.372524 23352 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:52:57.372670 23352 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.113.129
I20260809 22:52:57.381439 23352 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.113.148:38805
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786315935617919-15814-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786315935617919-15814-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.113.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786315935617919-15814-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.113.129
--webserver_port=0
--tserver_master_addrs=127.15.113.190:42223
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.113.129
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786315935617919-15814-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:52:57.382607 23352 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:52:57.384305 23352 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:52:57.397007 23358 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:52:57.397027 23364 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:52:57.397222 23360 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:52:57.397893 23352 server_base.cc:1034] running on GCE node
I20260809 22:52:57.398438 23352 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:52:57.399289 23352 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:52:57.400488 23352 hybrid_clock.cc:648] HybridClock initialized: now 1786315977400435 us; error 44 us; skew 500 ppm
I20260809 22:52:57.400774 23352 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 22:52:57.402910 23352 webserver.cc:460] Webserver started at http://127.15.113.129:44301/ using document root <none> and password file <none>
I20260809 22:52:57.403548 23352 fs_manager.cc:359] Metadata directory not provided
I20260809 22:52:57.403650 23352 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:52:57.403968 23352 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:52:57.406716 23352 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786315935617919-15814-0/minicluster-data/ts-0/data/instance:
uuid: "8f21b6c125c643d98b4116feaff8e4b1"
format_stamp: "Formatted at 2026-08-09 22:52:57 on dist-test-slave-ncp9"
server_key: "1eabad91fd6f2bd83e83898e301a068f"
server_key_iv: "d5e10c1e675ea6538e96665a91f7a355"
server_key_version: "encryptionkey@0"
I20260809 22:52:57.407472 23352 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786315935617919-15814-0/minicluster-data/ts-0/wal/instance:
uuid: "8f21b6c125c643d98b4116feaff8e4b1"
format_stamp: "Formatted at 2026-08-09 22:52:57 on dist-test-slave-ncp9"
server_key: "1eabad91fd6f2bd83e83898e301a068f"
server_key_iv: "d5e10c1e675ea6538e96665a91f7a355"
server_key_version: "encryptionkey@0"
I20260809 22:52:57.412274 23352 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260809 22:52:57.415472 23373 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:52:57.416671 23352 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 22:52:57.416815 23352 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786315935617919-15814-0/minicluster-data/ts-0/data,/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786315935617919-15814-0/minicluster-data/ts-0/wal
uuid: "8f21b6c125c643d98b4116feaff8e4b1"
format_stamp: "Formatted at 2026-08-09 22:52:57 on dist-test-slave-ncp9"
server_key: "1eabad91fd6f2bd83e83898e301a068f"
server_key_iv: "d5e10c1e675ea6538e96665a91f7a355"
server_key_version: "encryptionkey@0"
I20260809 22:52:57.417065 23352 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786315935617919-15814-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786315935617919-15814-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786315935617919-15814-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:52:57.422410 23352 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:52:57.423127 23352 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:52:57.423362 23352 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:52:57.424206 23352 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:52:57.425554 23352 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:52:57.425648 23352 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:52:57.425750 23352 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:52:57.425818 23352 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:52:57.457571 23352 rpc_server.cc:307] RPC server started. Bound to: 127.15.113.129:43463
I20260809 22:52:57.457614 23545 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.113.129:43463 every 8 connection(s)
I20260809 22:52:57.458918 23352 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786315935617919-15814-0/minicluster-data/ts-0/data/info.pb
I20260809 22:52:57.462605 15814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskItYPJR/build/asan/bin/kudu as pid 23352
I20260809 22:52:57.462807 15814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786315935617919-15814-0/minicluster-data/ts-0/wal/instance
I20260809 22:52:57.463660 15814 external_mini_cluster.cc:1428] Setting key 348187bbd74501f214a9a3a41a302ca5
I20260809 22:52:57.466694 15814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskItYPJR/build/asan/bin/kudu
/tmp/dist-test-taskItYPJR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786315935617919-15814-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786315935617919-15814-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786315935617919-15814-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786315935617919-15814-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.113.130:0
--local_ip_for_outbound_sockets=127.15.113.130
--webserver_interface=127.15.113.130
--webserver_port=0
--tserver_master_addrs=127.15.113.190:42223
--builtin_ntp_servers=127.15.113.148:38805
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260809 22:52:57.467535 23554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:52:57.461343 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.113.129:39745 (local address 127.15.113.190:42223)
0809 22:52:57.461793 (+ 450us) server_negotiation.cc:206] Beginning negotiation
0809 22:52:57.461800 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:52:57.463246 (+ 1446us) server_negotiation.cc:407] Connection header received
0809 22:52:57.465308 (+ 2062us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:52:57.465313 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:52:57.465366 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:52:57.465463 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:52:57.466797 (+ 1334us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:52:57.466803 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:52:57.466809 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 22:52:57.466828 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:52:57.466886 (+ 58us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 22:52:57.466904 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:52:57.467024 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0809 22:52:57.467176 (+ 152us) server_negotiation.cc:299] Negotiation successful
0809 22:52:57.467233 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":318,"thread_start_us":211,"threads_started":1}
I20260809 22:52:57.468293 23556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:52:57.461955 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:42223 (local address 127.15.113.129:39745)
0809 22:52:57.463071 (+ 1116us) negotiation.cc:107] Waiting for socket to connect
0809 22:52:57.463134 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0809 22:52:57.465048 (+ 1914us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:52:57.465713 (+ 665us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:52:57.465725 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:52:57.465842 (+ 117us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:52:57.466495 (+ 653us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 22:52:57.466498 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:52:57.466533 (+ 35us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 22:52:57.466616 (+ 83us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:52:57.466998 (+ 382us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:52:57.467007 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:52:57.467013 (+ 6us) client_negotiation.cc:769] Sending connection context
0809 22:52:57.467224 (+ 211us) client_negotiation.cc:240] Negotiation successful
0809 22:52:57.467404 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":516,"thread_start_us":294,"threads_started":1}
I20260809 22:52:57.469696 23549 heartbeater.cc:347] Connected to a master server at 127.15.113.190:42223
I20260809 22:52:57.469964 23549 heartbeater.cc:464] Registering TS with master...
I20260809 22:52:57.470530 23549 heartbeater.cc:511] Master 127.15.113.190:42223 requested a full tablet report, sending...
I20260809 22:52:57.472395 23262 ts_manager.cc:194] Registered new tserver with Master: 8f21b6c125c643d98b4116feaff8e4b1 (127.15.113.129:43463)
W20260809 22:52:57.791574 23557 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:52:57.791836 23557 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:52:57.791944 23557 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:52:57.798969 23557 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:52:57.799067 23557 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:52:57.799214 23557 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.113.130
I20260809 22:52:57.808082 23557 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.113.148:38805
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786315935617919-15814-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786315935617919-15814-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.113.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786315935617919-15814-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.113.130
--webserver_port=0
--tserver_master_addrs=127.15.113.190:42223
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.113.130
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786315935617919-15814-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:52:57.809315 23557 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:52:57.811071 23557 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:52:57.823285 23562 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:52:57.823343 23563 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:52:57.823769 23565 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:52:57.824406 23557 server_base.cc:1034] running on GCE node
I20260809 22:52:57.825043 23557 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:52:57.825907 23557 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:52:57.827122 23557 hybrid_clock.cc:648] HybridClock initialized: now 1786315977827077 us; error 31 us; skew 500 ppm
I20260809 22:52:57.827414 23557 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 22:52:57.829615 23557 webserver.cc:460] Webserver started at http://127.15.113.130:41127/ using document root <none> and password file <none>
I20260809 22:52:57.830253 23557 fs_manager.cc:359] Metadata directory not provided
I20260809 22:52:57.830351 23557 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:52:57.830673 23557 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:52:57.833446 23557 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786315935617919-15814-0/minicluster-data/ts-1/data/instance:
uuid: "f9db302e921d4b3b8865cb06f2b6a40e"
format_stamp: "Formatted at 2026-08-09 22:52:57 on dist-test-slave-ncp9"
server_key: "71aeecad643f3286053cb6d29d3b188b"
server_key_iv: "018bef662610fa0ff82ea44f238e79d4"
server_key_version: "encryptionkey@0"
I20260809 22:52:57.834221 23557 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786315935617919-15814-0/minicluster-data/ts-1/wal/instance:
uuid: "f9db302e921d4b3b8865cb06f2b6a40e"
format_stamp: "Formatted at 2026-08-09 22:52:57 on dist-test-slave-ncp9"
server_key: "71aeecad643f3286053cb6d29d3b188b"
server_key_iv: "018bef662610fa0ff82ea44f238e79d4"
server_key_version: "encryptionkey@0"
I20260809 22:52:57.839054 23557 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260809 22:52:57.842316 23575 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:52:57.843513 23557 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260809 22:52:57.843655 23557 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786315935617919-15814-0/minicluster-data/ts-1/data,/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786315935617919-15814-0/minicluster-data/ts-1/wal
uuid: "f9db302e921d4b3b8865cb06f2b6a40e"
format_stamp: "Formatted at 2026-08-09 22:52:57 on dist-test-slave-ncp9"
server_key: "71aeecad643f3286053cb6d29d3b188b"
server_key_iv: "018bef662610fa0ff82ea44f238e79d4"
server_key_version: "encryptionkey@0"
I20260809 22:52:57.843847 23557 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786315935617919-15814-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786315935617919-15814-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786315935617919-15814-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:52:57.849247 23557 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:52:57.849995 23557 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:52:57.850230 23557 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:52:57.851087 23557 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:52:57.852376 23557 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:52:57.852473 23557 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:52:57.852579 23557 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:52:57.852656 23557 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:52:57.881537 23557 rpc_server.cc:307] RPC server started. Bound to: 127.15.113.130:38919
I20260809 22:52:57.881596 23754 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.113.130:38919 every 8 connection(s)
I20260809 22:52:57.882766 23557 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786315935617919-15814-0/minicluster-data/ts-1/data/info.pb
I20260809 22:52:57.887951 15814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskItYPJR/build/asan/bin/kudu as pid 23557
I20260809 22:52:57.888108 15814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786315935617919-15814-0/minicluster-data/ts-1/wal/instance
I20260809 22:52:57.888599 15814 external_mini_cluster.cc:1428] Setting key 5b84c6874e1518ac2f169cf8b71132a1
I20260809 22:52:57.890829 23554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:52:57.885191 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.113.130:38687 (local address 127.15.113.190:42223)
0809 22:52:57.885368 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0809 22:52:57.885373 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:52:57.886761 (+ 1388us) server_negotiation.cc:407] Connection header received
0809 22:52:57.888544 (+ 1783us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:52:57.888550 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:52:57.888622 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:52:57.888738 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:52:57.890141 (+ 1403us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:52:57.890145 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:52:57.890149 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 22:52:57.890164 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:52:57.890196 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 22:52:57.890208 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:52:57.890304 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0809 22:52:57.890527 (+ 223us) server_negotiation.cc:299] Negotiation successful
0809 22:52:57.890578 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260809 22:52:57.891568 15814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskItYPJR/build/asan/bin/kudu
/tmp/dist-test-taskItYPJR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786315935617919-15814-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786315935617919-15814-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786315935617919-15814-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786315935617919-15814-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.113.131:0
--local_ip_for_outbound_sockets=127.15.113.131
--webserver_interface=127.15.113.131
--webserver_port=0
--tserver_master_addrs=127.15.113.190:42223
--builtin_ntp_servers=127.15.113.148:38805
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260809 22:52:57.891806 23770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:52:57.885717 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:42223 (local address 127.15.113.130:38687)
0809 22:52:57.886610 (+ 893us) negotiation.cc:107] Waiting for socket to connect
0809 22:52:57.886655 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0809 22:52:57.888289 (+ 1634us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:52:57.888967 (+ 678us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:52:57.888976 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:52:57.889066 (+ 90us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:52:57.889799 (+ 733us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 22:52:57.889802 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:52:57.889850 (+ 48us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 22:52:57.889957 (+ 107us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:52:57.890297 (+ 340us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:52:57.890308 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:52:57.890316 (+ 8us) client_negotiation.cc:769] Sending connection context
0809 22:52:57.890612 (+ 296us) client_negotiation.cc:240] Negotiation successful
0809 22:52:57.890841 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":447,"thread_start_us":263,"threads_started":1}
I20260809 22:52:57.893083 23759 heartbeater.cc:347] Connected to a master server at 127.15.113.190:42223
I20260809 22:52:57.893409 23759 heartbeater.cc:464] Registering TS with master...
I20260809 22:52:57.894043 23759 heartbeater.cc:511] Master 127.15.113.190:42223 requested a full tablet report, sending...
I20260809 22:52:57.895444 23262 ts_manager.cc:194] Registered new tserver with Master: f9db302e921d4b3b8865cb06f2b6a40e (127.15.113.130:38919)
W20260809 22:52:58.219677 23771 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:52:58.219975 23771 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:52:58.220075 23771 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:52:58.227138 23771 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:52:58.227231 23771 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:52:58.227344 23771 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.113.131
I20260809 22:52:58.236235 23771 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.113.148:38805
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786315935617919-15814-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786315935617919-15814-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.113.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786315935617919-15814-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.113.131
--webserver_port=0
--tserver_master_addrs=127.15.113.190:42223
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.113.131
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786315935617919-15814-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:52:58.237538 23771 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:52:58.239248 23771 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:52:58.251828 23783 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:52:58.251799 23786 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:52:58.251863 23782 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:52:58.252693 23771 server_base.cc:1034] running on GCE node
I20260809 22:52:58.253350 23771 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:52:58.254227 23771 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:52:58.255415 23771 hybrid_clock.cc:648] HybridClock initialized: now 1786315978255377 us; error 33 us; skew 500 ppm
I20260809 22:52:58.255709 23771 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 22:52:58.257814 23771 webserver.cc:460] Webserver started at http://127.15.113.131:44897/ using document root <none> and password file <none>
I20260809 22:52:58.258479 23771 fs_manager.cc:359] Metadata directory not provided
I20260809 22:52:58.258571 23771 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:52:58.258888 23771 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:52:58.261629 23771 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786315935617919-15814-0/minicluster-data/ts-2/data/instance:
uuid: "3f3c9b7780414c30891d9ea71f13bac6"
format_stamp: "Formatted at 2026-08-09 22:52:58 on dist-test-slave-ncp9"
server_key: "723f76e7b2ede93d2bde1de14810cb0e"
server_key_iv: "853ad0c07175d909728dccef7fc5ffa4"
server_key_version: "encryptionkey@0"
I20260809 22:52:58.262390 23771 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786315935617919-15814-0/minicluster-data/ts-2/wal/instance:
uuid: "3f3c9b7780414c30891d9ea71f13bac6"
format_stamp: "Formatted at 2026-08-09 22:52:58 on dist-test-slave-ncp9"
server_key: "723f76e7b2ede93d2bde1de14810cb0e"
server_key_iv: "853ad0c07175d909728dccef7fc5ffa4"
server_key_version: "encryptionkey@0"
I20260809 22:52:58.267251 23771 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 22:52:58.270489 23793 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:52:58.271704 23771 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260809 22:52:58.271848 23771 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786315935617919-15814-0/minicluster-data/ts-2/data,/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786315935617919-15814-0/minicluster-data/ts-2/wal
uuid: "3f3c9b7780414c30891d9ea71f13bac6"
format_stamp: "Formatted at 2026-08-09 22:52:58 on dist-test-slave-ncp9"
server_key: "723f76e7b2ede93d2bde1de14810cb0e"
server_key_iv: "853ad0c07175d909728dccef7fc5ffa4"
server_key_version: "encryptionkey@0"
I20260809 22:52:58.272034 23771 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786315935617919-15814-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786315935617919-15814-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786315935617919-15814-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:52:58.277366 23771 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:52:58.278079 23771 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:52:58.278321 23771 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:52:58.279136 23771 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:52:58.280400 23771 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:52:58.280490 23771 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:52:58.280591 23771 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:52:58.280660 23771 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:52:58.309087 23771 rpc_server.cc:307] RPC server started. Bound to: 127.15.113.131:37445
I20260809 22:52:58.309142 23985 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.113.131:37445 every 8 connection(s)
I20260809 22:52:58.310289 23771 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786315935617919-15814-0/minicluster-data/ts-2/data/info.pb
I20260809 22:52:58.314587 15814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskItYPJR/build/asan/bin/kudu as pid 23771
I20260809 22:52:58.314747 15814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786315935617919-15814-0/minicluster-data/ts-2/wal/instance
I20260809 22:52:58.315311 15814 external_mini_cluster.cc:1428] Setting key 58155ccd98c7c31701f437cb623ae124
I20260809 22:52:58.318403 23554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:52:58.312606 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.113.131:54017 (local address 127.15.113.190:42223)
0809 22:52:58.312785 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0809 22:52:58.312790 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:52:58.314158 (+ 1368us) server_negotiation.cc:407] Connection header received
0809 22:52:58.316266 (+ 2108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:52:58.316270 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:52:58.316324 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:52:58.316403 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:52:58.317723 (+ 1320us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:52:58.317727 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:52:58.317730 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 22:52:58.317749 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:52:58.317782 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 22:52:58.317794 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:52:58.317899 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0809 22:52:58.318078 (+ 179us) server_negotiation.cc:299] Negotiation successful
0809 22:52:58.318145 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260809 22:52:58.319268 23990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:52:58.313138 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:42223 (local address 127.15.113.131:54017)
0809 22:52:58.314018 (+ 880us) negotiation.cc:107] Waiting for socket to connect
0809 22:52:58.314054 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0809 22:52:58.316024 (+ 1970us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:52:58.316675 (+ 651us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:52:58.316683 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:52:58.316775 (+ 92us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:52:58.317436 (+ 661us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 22:52:58.317438 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:52:58.317475 (+ 37us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 22:52:58.317557 (+ 82us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:52:58.317879 (+ 322us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:52:58.317890 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:52:58.317898 (+ 8us) client_negotiation.cc:769] Sending connection context
0809 22:52:58.318155 (+ 257us) client_negotiation.cc:240] Negotiation successful
0809 22:52:58.318390 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":442,"thread_start_us":255,"threads_started":1}
I20260809 22:52:58.320399 23987 heartbeater.cc:347] Connected to a master server at 127.15.113.190:42223
I20260809 22:52:58.320670 23987 heartbeater.cc:464] Registering TS with master...
I20260809 22:52:58.321274 23987 heartbeater.cc:511] Master 127.15.113.190:42223 requested a full tablet report, sending...
I20260809 22:52:58.322539 23262 ts_manager.cc:194] Registered new tserver with Master: 3f3c9b7780414c30891d9ea71f13bac6 (127.15.113.131:37445)
I20260809 22:52:58.330001 15814 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 22:52:58.336429 23554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:52:58.334256 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55790 (local address 127.15.113.190:42223)
0809 22:52:58.334399 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0809 22:52:58.334403 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 22:52:58.334886 (+ 483us) server_negotiation.cc:407] Connection header received
0809 22:52:58.335014 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:52:58.335018 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:52:58.335069 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:52:58.335135 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:52:58.335935 (+ 800us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:52:58.335938 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:52:58.335941 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 22:52:58.335953 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:52:58.335977 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 22:52:58.335987 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:52:58.336074 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0809 22:52:58.336136 (+ 62us) server_negotiation.cc:299] Negotiation successful
0809 22:52:58.336178 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260809 22:52:58.339910 23262 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:55790:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 22:52:58.346462 23262 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 22:52:58.371858 24013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:52:58.358252 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39088 (local address 127.15.113.130:38919)
0809 22:52:58.368504 (+ 10252us) server_negotiation.cc:206] Beginning negotiation
0809 22:52:58.368513 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 22:52:58.369154 (+ 641us) server_negotiation.cc:407] Connection header received
0809 22:52:58.369240 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:52:58.369249 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:52:58.369495 (+ 246us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:52:58.369668 (+ 173us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:52:58.370968 (+ 1300us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:52:58.370977 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:52:58.370991 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 22:52:58.371011 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:52:58.371076 (+ 65us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 22:52:58.371099 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:52:58.371298 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0809 22:52:58.371422 (+ 124us) server_negotiation.cc:299] Negotiation successful
0809 22:52:58.371532 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10062,"thread_start_us":9998,"threads_started":1}
I20260809 22:52:58.371978 24014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:52:58.357866 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.130:38919 (local address 127.0.0.1:39088)
0809 22:52:58.368878 (+ 11012us) negotiation.cc:107] Waiting for socket to connect
0809 22:52:58.368921 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 22:52:58.369039 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:52:58.369690 (+ 651us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:52:58.369697 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:52:58.369792 (+ 95us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:52:58.370700 (+ 908us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 22:52:58.370704 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:52:58.370737 (+ 33us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 22:52:58.370804 (+ 67us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:52:58.371230 (+ 426us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:52:58.371243 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:52:58.371252 (+ 9us) client_negotiation.cc:769] Sending connection context
0809 22:52:58.371433 (+ 181us) client_negotiation.cc:240] Negotiation successful
0809 22:52:58.371521 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10852,"thread_start_us":11048,"threads_started":1}
I20260809 22:52:58.372689 24007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:52:58.357639 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.131:37445 (local address 127.0.0.1:48408)
0809 22:52:58.359611 (+ 1972us) negotiation.cc:107] Waiting for socket to connect
0809 22:52:58.359670 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0809 22:52:58.359812 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:52:58.370101 (+ 10289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:52:58.370106 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:52:58.370142 (+ 36us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:52:58.370697 (+ 555us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 22:52:58.370701 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:52:58.370735 (+ 34us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 22:52:58.370805 (+ 70us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:52:58.372131 (+ 1326us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:52:58.372142 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:52:58.372147 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 22:52:58.372262 (+ 115us) client_negotiation.cc:240] Negotiation successful
0809 22:52:58.372311 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1705,"thread_start_us":1541,"threads_started":1}
I20260809 22:52:58.373428 24011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:52:58.360008 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.129:43463 (local address 127.0.0.1:38688)
0809 22:52:58.369200 (+ 9192us) negotiation.cc:107] Waiting for socket to connect
0809 22:52:58.369224 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0809 22:52:58.369326 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:52:58.370704 (+ 1378us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:52:58.370708 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:52:58.370730 (+ 22us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:52:58.372187 (+ 1457us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 22:52:58.372190 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:52:58.372205 (+ 15us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 22:52:58.372249 (+ 44us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:52:58.372616 (+ 367us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:52:58.372629 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:52:58.372634 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 22:52:58.372950 (+ 316us) client_negotiation.cc:240] Negotiation successful
0809 22:52:58.373006 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9062,"thread_start_us":8566,"threads_started":1}
I20260809 22:52:58.373505 24010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:52:58.357786 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48408 (local address 127.15.113.131:37445)
0809 22:52:58.369613 (+ 11827us) server_negotiation.cc:206] Beginning negotiation
0809 22:52:58.369620 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:52:58.369653 (+ 33us) server_negotiation.cc:407] Connection header received
0809 22:52:58.369716 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:52:58.369723 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:52:58.369908 (+ 185us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:52:58.370520 (+ 612us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:52:58.371884 (+ 1364us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:52:58.371893 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:52:58.371907 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 22:52:58.371922 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:52:58.371973 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 22:52:58.371993 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:52:58.372980 (+ 987us) server_negotiation.cc:1035] Waiting for connection context
0809 22:52:58.373048 (+ 68us) server_negotiation.cc:299] Negotiation successful
0809 22:52:58.373151 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11665,"thread_start_us":11501,"threads_started":1}
I20260809 22:52:58.373402 24015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:52:58.360340 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38688 (local address 127.15.113.129:43463)
0809 22:52:58.368340 (+ 8000us) server_negotiation.cc:206] Beginning negotiation
0809 22:52:58.368349 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 22:52:58.369443 (+ 1094us) server_negotiation.cc:407] Connection header received
0809 22:52:58.369523 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:52:58.369529 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:52:58.369729 (+ 200us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:52:58.369873 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:52:58.372369 (+ 2496us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:52:58.372375 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:52:58.372389 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 22:52:58.372408 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:52:58.372473 (+ 65us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 22:52:58.372500 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:52:58.372641 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0809 22:52:58.372766 (+ 125us) server_negotiation.cc:299] Negotiation successful
0809 22:52:58.373113 (+ 347us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7840,"thread_start_us":7716,"threads_started":1}
I20260809 22:52:58.376271 23646 tablet_service.cc:1467] Processing CreateTablet for tablet 9b6c76f9ce3e4e64aaf6c3e58225b06e (DEFAULT_TABLE table=test-table [id=63b08e6c5c26463fbab442bfba6522e3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:52:58.377046 23879 tablet_service.cc:1467] Processing CreateTablet for tablet 9b6c76f9ce3e4e64aaf6c3e58225b06e (DEFAULT_TABLE table=test-table [id=63b08e6c5c26463fbab442bfba6522e3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:52:58.377875 23646 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9b6c76f9ce3e4e64aaf6c3e58225b06e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:52:58.378176 23879 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9b6c76f9ce3e4e64aaf6c3e58225b06e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:52:58.378206 23449 tablet_service.cc:1467] Processing CreateTablet for tablet 9b6c76f9ce3e4e64aaf6c3e58225b06e (DEFAULT_TABLE table=test-table [id=63b08e6c5c26463fbab442bfba6522e3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:52:58.379429 23449 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9b6c76f9ce3e4e64aaf6c3e58225b06e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:52:58.390066 24017 tablet_bootstrap.cc:492] T 9b6c76f9ce3e4e64aaf6c3e58225b06e P 3f3c9b7780414c30891d9ea71f13bac6: Bootstrap starting.
I20260809 22:52:58.393049 24017 tablet_bootstrap.cc:654] T 9b6c76f9ce3e4e64aaf6c3e58225b06e P 3f3c9b7780414c30891d9ea71f13bac6: Neither blocks nor log segments found. Creating new log.
I20260809 22:52:58.393442 24018 tablet_bootstrap.cc:492] T 9b6c76f9ce3e4e64aaf6c3e58225b06e P f9db302e921d4b3b8865cb06f2b6a40e: Bootstrap starting.
I20260809 22:52:58.394393 24017 log.cc:824] T 9b6c76f9ce3e4e64aaf6c3e58225b06e P 3f3c9b7780414c30891d9ea71f13bac6: Log is configured to *not* fsync() on all Append() calls
I20260809 22:52:58.395933 24021 tablet_bootstrap.cc:492] T 9b6c76f9ce3e4e64aaf6c3e58225b06e P 8f21b6c125c643d98b4116feaff8e4b1: Bootstrap starting.
I20260809 22:52:58.396790 24018 tablet_bootstrap.cc:654] T 9b6c76f9ce3e4e64aaf6c3e58225b06e P f9db302e921d4b3b8865cb06f2b6a40e: Neither blocks nor log segments found. Creating new log.
I20260809 22:52:58.397713 24017 tablet_bootstrap.cc:492] T 9b6c76f9ce3e4e64aaf6c3e58225b06e P 3f3c9b7780414c30891d9ea71f13bac6: No bootstrap required, opened a new log
I20260809 22:52:58.398031 24017 ts_tablet_manager.cc:1397] T 9b6c76f9ce3e4e64aaf6c3e58225b06e P 3f3c9b7780414c30891d9ea71f13bac6: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260809 22:52:58.398126 24018 log.cc:824] T 9b6c76f9ce3e4e64aaf6c3e58225b06e P f9db302e921d4b3b8865cb06f2b6a40e: Log is configured to *not* fsync() on all Append() calls
I20260809 22:52:58.399281 24021 tablet_bootstrap.cc:654] T 9b6c76f9ce3e4e64aaf6c3e58225b06e P 8f21b6c125c643d98b4116feaff8e4b1: Neither blocks nor log segments found. Creating new log.
I20260809 22:52:58.400460 24021 log.cc:824] T 9b6c76f9ce3e4e64aaf6c3e58225b06e P 8f21b6c125c643d98b4116feaff8e4b1: Log is configured to *not* fsync() on all Append() calls
I20260809 22:52:58.401221 24018 tablet_bootstrap.cc:492] T 9b6c76f9ce3e4e64aaf6c3e58225b06e P f9db302e921d4b3b8865cb06f2b6a40e: No bootstrap required, opened a new log
I20260809 22:52:58.401448 24018 ts_tablet_manager.cc:1397] T 9b6c76f9ce3e4e64aaf6c3e58225b06e P f9db302e921d4b3b8865cb06f2b6a40e: Time spent bootstrapping tablet: real 0.008s user 0.004s sys 0.003s
I20260809 22:52:58.403347 24021 tablet_bootstrap.cc:492] T 9b6c76f9ce3e4e64aaf6c3e58225b06e P 8f21b6c125c643d98b4116feaff8e4b1: No bootstrap required, opened a new log
I20260809 22:52:58.403594 24021 ts_tablet_manager.cc:1397] T 9b6c76f9ce3e4e64aaf6c3e58225b06e P 8f21b6c125c643d98b4116feaff8e4b1: Time spent bootstrapping tablet: real 0.008s user 0.004s sys 0.002s
I20260809 22:52:58.405375 24017 raft_consensus.cc:348] T 9b6c76f9ce3e4e64aaf6c3e58225b06e P 3f3c9b7780414c30891d9ea71f13bac6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f3c9b7780414c30891d9ea71f13bac6" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 37445 } } peers { permanent_uuid: "f9db302e921d4b3b8865cb06f2b6a40e" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 38919 } } peers { permanent_uuid: "8f21b6c125c643d98b4116feaff8e4b1" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 43463 } }
I20260809 22:52:58.405694 24017 raft_consensus.cc:374] T 9b6c76f9ce3e4e64aaf6c3e58225b06e P 3f3c9b7780414c30891d9ea71f13bac6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:52:58.405825 24017 raft_consensus.cc:729] T 9b6c76f9ce3e4e64aaf6c3e58225b06e P 3f3c9b7780414c30891d9ea71f13bac6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f3c9b7780414c30891d9ea71f13bac6, State: Initialized, Role: FOLLOWER
I20260809 22:52:58.406363 24017 consensus_queue.cc:260] T 9b6c76f9ce3e4e64aaf6c3e58225b06e P 3f3c9b7780414c30891d9ea71f13bac6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f3c9b7780414c30891d9ea71f13bac6" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 37445 } } peers { permanent_uuid: "f9db302e921d4b3b8865cb06f2b6a40e" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 38919 } } peers { permanent_uuid: "8f21b6c125c643d98b4116feaff8e4b1" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 43463 } }
I20260809 22:52:58.407353 23987 heartbeater.cc:503] Master 127.15.113.190:42223 was elected leader, sending a full tablet report...
I20260809 22:52:58.407769 24017 ts_tablet_manager.cc:1428] T 9b6c76f9ce3e4e64aaf6c3e58225b06e P 3f3c9b7780414c30891d9ea71f13bac6: Time spent starting tablet: real 0.010s user 0.009s sys 0.000s
I20260809 22:52:58.412873 24018 raft_consensus.cc:348] T 9b6c76f9ce3e4e64aaf6c3e58225b06e P f9db302e921d4b3b8865cb06f2b6a40e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f3c9b7780414c30891d9ea71f13bac6" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 37445 } } peers { permanent_uuid: "f9db302e921d4b3b8865cb06f2b6a40e" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 38919 } } peers { permanent_uuid: "8f21b6c125c643d98b4116feaff8e4b1" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 43463 } }
I20260809 22:52:58.413345 24018 raft_consensus.cc:374] T 9b6c76f9ce3e4e64aaf6c3e58225b06e P f9db302e921d4b3b8865cb06f2b6a40e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:52:58.413445 24018 raft_consensus.cc:729] T 9b6c76f9ce3e4e64aaf6c3e58225b06e P f9db302e921d4b3b8865cb06f2b6a40e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f9db302e921d4b3b8865cb06f2b6a40e, State: Initialized, Role: FOLLOWER
I20260809 22:52:58.413552 24021 raft_consensus.cc:348] T 9b6c76f9ce3e4e64aaf6c3e58225b06e P 8f21b6c125c643d98b4116feaff8e4b1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f3c9b7780414c30891d9ea71f13bac6" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 37445 } } peers { permanent_uuid: "f9db302e921d4b3b8865cb06f2b6a40e" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 38919 } } peers { permanent_uuid: "8f21b6c125c643d98b4116feaff8e4b1" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 43463 } }
I20260809 22:52:58.413933 24021 raft_consensus.cc:374] T 9b6c76f9ce3e4e64aaf6c3e58225b06e P 8f21b6c125c643d98b4116feaff8e4b1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:52:58.414029 24021 raft_consensus.cc:729] T 9b6c76f9ce3e4e64aaf6c3e58225b06e P 8f21b6c125c643d98b4116feaff8e4b1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f21b6c125c643d98b4116feaff8e4b1, State: Initialized, Role: FOLLOWER
I20260809 22:52:58.414059 24018 consensus_queue.cc:260] T 9b6c76f9ce3e4e64aaf6c3e58225b06e P f9db302e921d4b3b8865cb06f2b6a40e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f3c9b7780414c30891d9ea71f13bac6" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 37445 } } peers { permanent_uuid: "f9db302e921d4b3b8865cb06f2b6a40e" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 38919 } } peers { permanent_uuid: "8f21b6c125c643d98b4116feaff8e4b1" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 43463 } }
I20260809 22:52:58.414623 24021 consensus_queue.cc:260] T 9b6c76f9ce3e4e64aaf6c3e58225b06e P 8f21b6c125c643d98b4116feaff8e4b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f3c9b7780414c30891d9ea71f13bac6" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 37445 } } peers { permanent_uuid: "f9db302e921d4b3b8865cb06f2b6a40e" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 38919 } } peers { permanent_uuid: "8f21b6c125c643d98b4116feaff8e4b1" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 43463 } }
I20260809 22:52:58.414999 23759 heartbeater.cc:503] Master 127.15.113.190:42223 was elected leader, sending a full tablet report...
I20260809 22:52:58.415688 23549 heartbeater.cc:503] Master 127.15.113.190:42223 was elected leader, sending a full tablet report...
I20260809 22:52:58.415735 24018 ts_tablet_manager.cc:1428] T 9b6c76f9ce3e4e64aaf6c3e58225b06e P f9db302e921d4b3b8865cb06f2b6a40e: Time spent starting tablet: real 0.014s user 0.005s sys 0.007s
I20260809 22:52:58.417163 24021 ts_tablet_manager.cc:1428] T 9b6c76f9ce3e4e64aaf6c3e58225b06e P 8f21b6c125c643d98b4116feaff8e4b1: Time spent starting tablet: real 0.013s user 0.008s sys 0.006s
W20260809 22:52:58.461025 23552 tablet.cc:2369] T 9b6c76f9ce3e4e64aaf6c3e58225b06e P 8f21b6c125c643d98b4116feaff8e4b1: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 22:52:58.561851 23989 tablet.cc:2369] T 9b6c76f9ce3e4e64aaf6c3e58225b06e P 3f3c9b7780414c30891d9ea71f13bac6: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 22:52:58.565227 24028 raft_consensus.cc:482] T 9b6c76f9ce3e4e64aaf6c3e58225b06e P 8f21b6c125c643d98b4116feaff8e4b1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 22:52:58.565502 24028 raft_consensus.cc:504] T 9b6c76f9ce3e4e64aaf6c3e58225b06e P 8f21b6c125c643d98b4116feaff8e4b1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f3c9b7780414c30891d9ea71f13bac6" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 37445 } } peers { permanent_uuid: "f9db302e921d4b3b8865cb06f2b6a40e" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 38919 } } peers { permanent_uuid: "8f21b6c125c643d98b4116feaff8e4b1" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 43463 } }
I20260809 22:52:58.567096 24028 leader_election.cc:288] T 9b6c76f9ce3e4e64aaf6c3e58225b06e P 8f21b6c125c643d98b4116feaff8e4b1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3f3c9b7780414c30891d9ea71f13bac6 (127.15.113.131:37445), f9db302e921d4b3b8865cb06f2b6a40e (127.15.113.130:38919)
I20260809 22:52:58.570421 24010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:52:58.567564 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.113.129:37335 (local address 127.15.113.131:37445)
0809 22:52:58.567772 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0809 22:52:58.567779 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:52:58.568562 (+ 783us) server_negotiation.cc:407] Connection header received
0809 22:52:58.568699 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:52:58.568703 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:52:58.568752 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:52:58.568833 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:52:58.569849 (+ 1016us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:52:58.569853 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:52:58.569856 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 22:52:58.569871 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:52:58.569901 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 22:52:58.569912 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:52:58.570020 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0809 22:52:58.570113 (+ 93us) server_negotiation.cc:299] Negotiation successful
0809 22:52:58.570172 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260809 22:52:58.570515 24013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:52:58.567657 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.113.129:50245 (local address 127.15.113.130:38919)
0809 22:52:58.567847 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0809 22:52:58.567852 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:52:58.568505 (+ 653us) server_negotiation.cc:407] Connection header received
0809 22:52:58.568664 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:52:58.568668 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:52:58.568717 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:52:58.568799 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:52:58.569921 (+ 1122us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:52:58.569927 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:52:58.569932 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 22:52:58.569949 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:52:58.569982 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 22:52:58.569994 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:52:58.570101 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0809 22:52:58.570202 (+ 101us) server_negotiation.cc:299] Negotiation successful
0809 22:52:58.570264 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260809 22:52:58.570719 24031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:52:58.567603 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.131:37445 (local address 127.15.113.129:37335)
0809 22:52:58.568458 (+ 855us) negotiation.cc:107] Waiting for socket to connect
0809 22:52:58.568472 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 22:52:58.568583 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:52:58.568876 (+ 293us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:52:58.568881 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:52:58.568933 (+ 52us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:52:58.569657 (+ 724us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 22:52:58.569659 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:52:58.569672 (+ 13us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 22:52:58.569713 (+ 41us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:52:58.569997 (+ 284us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:52:58.570002 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:52:58.570006 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 22:52:58.570156 (+ 150us) client_negotiation.cc:240] Negotiation successful
0809 22:52:58.570206 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":748,"mutex_wait_us":52,"thread_start_us":459,"threads_started":1}
I20260809 22:52:58.570755 24030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:52:58.567600 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.130:38919 (local address 127.15.113.129:50245)
0809 22:52:58.568387 (+ 787us) negotiation.cc:107] Waiting for socket to connect
0809 22:52:58.568406 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 22:52:58.568532 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:52:58.568849 (+ 317us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:52:58.568853 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:52:58.568876 (+ 23us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:52:58.569731 (+ 855us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 22:52:58.569734 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:52:58.569746 (+ 12us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 22:52:58.569782 (+ 36us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:52:58.570078 (+ 296us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:52:58.570083 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:52:58.570087 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 22:52:58.570250 (+ 163us) client_negotiation.cc:240] Negotiation successful
0809 22:52:58.570295 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":673,"thread_start_us":409,"threads_started":1}
I20260809 22:52:58.571820 23910 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9b6c76f9ce3e4e64aaf6c3e58225b06e" candidate_uuid: "8f21b6c125c643d98b4116feaff8e4b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f3c9b7780414c30891d9ea71f13bac6" is_pre_election: true
I20260809 22:52:58.571820 23681 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9b6c76f9ce3e4e64aaf6c3e58225b06e" candidate_uuid: "8f21b6c125c643d98b4116feaff8e4b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f9db302e921d4b3b8865cb06f2b6a40e" is_pre_election: true
I20260809 22:52:58.572357 23910 raft_consensus.cc:2454] T 9b6c76f9ce3e4e64aaf6c3e58225b06e P 3f3c9b7780414c30891d9ea71f13bac6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f21b6c125c643d98b4116feaff8e4b1 in term 0.
I20260809 22:52:58.572357 23681 raft_consensus.cc:2454] T 9b6c76f9ce3e4e64aaf6c3e58225b06e P f9db302e921d4b3b8865cb06f2b6a40e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f21b6c125c643d98b4116feaff8e4b1 in term 0.
I20260809 22:52:58.573035 23378 leader_election.cc:302] T 9b6c76f9ce3e4e64aaf6c3e58225b06e P 8f21b6c125c643d98b4116feaff8e4b1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8f21b6c125c643d98b4116feaff8e4b1, f9db302e921d4b3b8865cb06f2b6a40e; no voters:
I20260809 22:52:58.573410 24028 raft_consensus.cc:2781] T 9b6c76f9ce3e4e64aaf6c3e58225b06e P 8f21b6c125c643d98b4116feaff8e4b1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 22:52:58.573555 24028 raft_consensus.cc:482] T 9b6c76f9ce3e4e64aaf6c3e58225b06e P 8f21b6c125c643d98b4116feaff8e4b1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 22:52:58.573662 24028 raft_consensus.cc:3037] T 9b6c76f9ce3e4e64aaf6c3e58225b06e P 8f21b6c125c643d98b4116feaff8e4b1 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:52:58.575764 24028 raft_consensus.cc:504] T 9b6c76f9ce3e4e64aaf6c3e58225b06e P 8f21b6c125c643d98b4116feaff8e4b1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f3c9b7780414c30891d9ea71f13bac6" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 37445 } } peers { permanent_uuid: "f9db302e921d4b3b8865cb06f2b6a40e" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 38919 } } peers { permanent_uuid: "8f21b6c125c643d98b4116feaff8e4b1" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 43463 } }
I20260809 22:52:58.576259 24028 leader_election.cc:288] T 9b6c76f9ce3e4e64aaf6c3e58225b06e P 8f21b6c125c643d98b4116feaff8e4b1 [CANDIDATE]: Term 1 election: Requested vote from peers 3f3c9b7780414c30891d9ea71f13bac6 (127.15.113.131:37445), f9db302e921d4b3b8865cb06f2b6a40e (127.15.113.130:38919)
I20260809 22:52:58.576640 23910 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9b6c76f9ce3e4e64aaf6c3e58225b06e" candidate_uuid: "8f21b6c125c643d98b4116feaff8e4b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f3c9b7780414c30891d9ea71f13bac6"
I20260809 22:52:58.576740 23681 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9b6c76f9ce3e4e64aaf6c3e58225b06e" candidate_uuid: "8f21b6c125c643d98b4116feaff8e4b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f9db302e921d4b3b8865cb06f2b6a40e"
I20260809 22:52:58.576822 23910 raft_consensus.cc:3037] T 9b6c76f9ce3e4e64aaf6c3e58225b06e P 3f3c9b7780414c30891d9ea71f13bac6 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:52:58.576963 23681 raft_consensus.cc:3037] T 9b6c76f9ce3e4e64aaf6c3e58225b06e P f9db302e921d4b3b8865cb06f2b6a40e [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:52:58.578938 23910 raft_consensus.cc:2454] T 9b6c76f9ce3e4e64aaf6c3e58225b06e P 3f3c9b7780414c30891d9ea71f13bac6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f21b6c125c643d98b4116feaff8e4b1 in term 1.
I20260809 22:52:58.579218 23681 raft_consensus.cc:2454] T 9b6c76f9ce3e4e64aaf6c3e58225b06e P f9db302e921d4b3b8865cb06f2b6a40e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f21b6c125c643d98b4116feaff8e4b1 in term 1.
I20260809 22:52:58.579267 23379 leader_election.cc:302] T 9b6c76f9ce3e4e64aaf6c3e58225b06e P 8f21b6c125c643d98b4116feaff8e4b1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f3c9b7780414c30891d9ea71f13bac6, 8f21b6c125c643d98b4116feaff8e4b1; no voters:
I20260809 22:52:58.579499 24028 raft_consensus.cc:2781] T 9b6c76f9ce3e4e64aaf6c3e58225b06e P 8f21b6c125c643d98b4116feaff8e4b1 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 22:52:58.579897 24028 raft_consensus.cc:686] T 9b6c76f9ce3e4e64aaf6c3e58225b06e P 8f21b6c125c643d98b4116feaff8e4b1 [term 1 LEADER]: Becoming Leader. State: Replica: 8f21b6c125c643d98b4116feaff8e4b1, State: Running, Role: LEADER
I20260809 22:52:58.580327 24028 consensus_queue.cc:237] T 9b6c76f9ce3e4e64aaf6c3e58225b06e P 8f21b6c125c643d98b4116feaff8e4b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f3c9b7780414c30891d9ea71f13bac6" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 37445 } } peers { permanent_uuid: "f9db302e921d4b3b8865cb06f2b6a40e" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 38919 } } peers { permanent_uuid: "8f21b6c125c643d98b4116feaff8e4b1" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 43463 } }
I20260809 22:52:58.584913 23261 catalog_manager.cc:5495] T 9b6c76f9ce3e4e64aaf6c3e58225b06e P 8f21b6c125c643d98b4116feaff8e4b1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8f21b6c125c643d98b4116feaff8e4b1 (127.15.113.129). New cstate: current_term: 1 leader_uuid: "8f21b6c125c643d98b4116feaff8e4b1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f3c9b7780414c30891d9ea71f13bac6" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 37445 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f9db302e921d4b3b8865cb06f2b6a40e" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 38919 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8f21b6c125c643d98b4116feaff8e4b1" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 43463 } health_report { overall_health: HEALTHY } } }
I20260809 22:52:58.600534 24015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:52:58.598486 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38690 (local address 127.15.113.129:43463)
0809 22:52:58.598646 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0809 22:52:58.598653 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:52:58.598800 (+ 147us) server_negotiation.cc:407] Connection header received
0809 22:52:58.598997 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:52:58.599001 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:52:58.599074 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:52:58.599171 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:52:58.599957 (+ 786us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:52:58.599963 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:52:58.599967 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 22:52:58.599983 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:52:58.600015 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 22:52:58.600026 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:52:58.600128 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0809 22:52:58.600223 (+ 95us) server_negotiation.cc:299] Negotiation successful
0809 22:52:58.600278 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260809 22:52:58.607858 23681 raft_consensus.cc:1261] T 9b6c76f9ce3e4e64aaf6c3e58225b06e P f9db302e921d4b3b8865cb06f2b6a40e [term 1 FOLLOWER]: Refusing update from remote peer 8f21b6c125c643d98b4116feaff8e4b1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 22:52:58.607885 23910 raft_consensus.cc:1261] T 9b6c76f9ce3e4e64aaf6c3e58225b06e P 3f3c9b7780414c30891d9ea71f13bac6 [term 1 FOLLOWER]: Refusing update from remote peer 8f21b6c125c643d98b4116feaff8e4b1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 22:52:58.608752 24032 consensus_queue.cc:1035] T 9b6c76f9ce3e4e64aaf6c3e58225b06e P 8f21b6c125c643d98b4116feaff8e4b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f3c9b7780414c30891d9ea71f13bac6" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 37445 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 22:52:58.609093 24028 consensus_queue.cc:1035] T 9b6c76f9ce3e4e64aaf6c3e58225b06e P 8f21b6c125c643d98b4116feaff8e4b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f9db302e921d4b3b8865cb06f2b6a40e" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 38919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 22:52:58.633134 23261 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:55790:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 22:52:58.633371 23261 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:55790:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 22:52:58.636435 23261 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 71056d810a3c42a99a5382ad97d48c18: Sending DeleteTablet for 3 replicas of tablet 9b6c76f9ce3e4e64aaf6c3e58225b06e
I20260809 22:52:58.637562 23646 tablet_service.cc:1514] Processing DeleteTablet for tablet 9b6c76f9ce3e4e64aaf6c3e58225b06e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:52:58 UTC) from {username='slave'} at 127.0.0.1:39088
I20260809 22:52:58.637562 23879 tablet_service.cc:1514] Processing DeleteTablet for tablet 9b6c76f9ce3e4e64aaf6c3e58225b06e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:52:58 UTC) from {username='slave'} at 127.0.0.1:48408
I20260809 22:52:58.638636 23449 tablet_service.cc:1514] Processing DeleteTablet for tablet 9b6c76f9ce3e4e64aaf6c3e58225b06e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:52:58 UTC) from {username='slave'} at 127.0.0.1:38688
I20260809 22:52:58.639411 15814 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskItYPJR/build/asan/bin/kudu pid 23352
I20260809 22:52:58.654116 24015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:52:58.640581 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38704 (local address 127.15.113.129:43463)
0809 22:52:58.640769 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0809 22:52:58.640775 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:52:58.651472 (+ 10697us) server_negotiation.cc:407] Connection header received
0809 22:52:58.651926 (+ 454us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:52:58.651932 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:52:58.651994 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:52:58.652073 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:52:58.653288 (+ 1215us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:52:58.653293 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:52:58.653298 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 22:52:58.653322 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:52:58.653360 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 22:52:58.653377 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:52:58.653467 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0809 22:52:58.653742 (+ 275us) server_negotiation.cc:299] Negotiation successful
0809 22:52:58.653804 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260809 22:52:58.655304 23541 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:52:58.657145 24052 tablet_replica.cc:330] T 9b6c76f9ce3e4e64aaf6c3e58225b06e P 3f3c9b7780414c30891d9ea71f13bac6: stopping tablet replica
I20260809 22:52:58.657629 24052 raft_consensus.cc:2229] T 9b6c76f9ce3e4e64aaf6c3e58225b06e P 3f3c9b7780414c30891d9ea71f13bac6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 22:52:58.658179 24052 raft_consensus.cc:2258] T 9b6c76f9ce3e4e64aaf6c3e58225b06e P 3f3c9b7780414c30891d9ea71f13bac6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 22:52:58.658979 24050 tablet_replica.cc:330] T 9b6c76f9ce3e4e64aaf6c3e58225b06e P f9db302e921d4b3b8865cb06f2b6a40e: stopping tablet replica
I20260809 22:52:58.659453 24050 raft_consensus.cc:2229] T 9b6c76f9ce3e4e64aaf6c3e58225b06e P f9db302e921d4b3b8865cb06f2b6a40e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 22:52:58.659960 24050 raft_consensus.cc:2258] T 9b6c76f9ce3e4e64aaf6c3e58225b06e P f9db302e921d4b3b8865cb06f2b6a40e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 22:52:58.660066 24052 ts_tablet_manager.cc:1905] T 9b6c76f9ce3e4e64aaf6c3e58225b06e P 3f3c9b7780414c30891d9ea71f13bac6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 22:52:58.662055 24050 ts_tablet_manager.cc:1905] T 9b6c76f9ce3e4e64aaf6c3e58225b06e P f9db302e921d4b3b8865cb06f2b6a40e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 22:52:58.667107 24052 ts_tablet_manager.cc:1918] T 9b6c76f9ce3e4e64aaf6c3e58225b06e P 3f3c9b7780414c30891d9ea71f13bac6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 22:52:58.667266 24052 log.cc:1196] T 9b6c76f9ce3e4e64aaf6c3e58225b06e P 3f3c9b7780414c30891d9ea71f13bac6: Deleting WAL directory at /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786315935617919-15814-0/minicluster-data/ts-2/wal/wals/9b6c76f9ce3e4e64aaf6c3e58225b06e
I20260809 22:52:58.667248 24050 ts_tablet_manager.cc:1918] T 9b6c76f9ce3e4e64aaf6c3e58225b06e P f9db302e921d4b3b8865cb06f2b6a40e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 22:52:58.667397 24050 log.cc:1196] T 9b6c76f9ce3e4e64aaf6c3e58225b06e P f9db302e921d4b3b8865cb06f2b6a40e: Deleting WAL directory at /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786315935617919-15814-0/minicluster-data/ts-1/wal/wals/9b6c76f9ce3e4e64aaf6c3e58225b06e
I20260809 22:52:58.667797 24050 ts_tablet_manager.cc:1939] T 9b6c76f9ce3e4e64aaf6c3e58225b06e P f9db302e921d4b3b8865cb06f2b6a40e: Deleting consensus metadata
I20260809 22:52:58.667796 24052 ts_tablet_manager.cc:1939] T 9b6c76f9ce3e4e64aaf6c3e58225b06e P 3f3c9b7780414c30891d9ea71f13bac6: Deleting consensus metadata
I20260809 22:52:58.669205 23231 catalog_manager.cc:4841] TS f9db302e921d4b3b8865cb06f2b6a40e (127.15.113.130:38919): tablet 9b6c76f9ce3e4e64aaf6c3e58225b06e (table test-table [id=63b08e6c5c26463fbab442bfba6522e3]) successfully deleted
I20260809 22:52:58.669507 23232 catalog_manager.cc:4841] TS 3f3c9b7780414c30891d9ea71f13bac6 (127.15.113.131:37445): tablet 9b6c76f9ce3e4e64aaf6c3e58225b06e (table test-table [id=63b08e6c5c26463fbab442bfba6522e3]) successfully deleted
I20260809 22:52:58.754192 24053 tablet_replica.cc:330] T 9b6c76f9ce3e4e64aaf6c3e58225b06e P 8f21b6c125c643d98b4116feaff8e4b1: stopping tablet replica
I20260809 22:52:58.754559 15814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskItYPJR/build/asan/bin/kudu with pid 23352
I20260809 22:52:58.778772 15814 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskItYPJR/build/asan/bin/kudu pid 23557
W20260809 22:52:58.778700 23232 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.15.113.129:43463 (error 108)
I20260809 22:52:58.780495 24011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:52:58.780076 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.129:43463 (local address 127.0.0.1:56616)
0809 22:52:58.780249 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0809 22:52:58.780365 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.113.129:43463: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260809 22:52:58.780731 23232 catalog_manager.cc:4584] TS 8f21b6c125c643d98b4116feaff8e4b1 (127.15.113.129:43463): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 9b6c76f9ce3e4e64aaf6c3e58225b06e: Network error: Client connection negotiation failed: client connection to 127.15.113.129:43463: connect: Connection refused (error 111)
I20260809 22:52:58.781710 24013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:52:58.779735 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55472 (local address 127.15.113.130:38919)
0809 22:52:58.779946 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0809 22:52:58.779950 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 22:52:58.779965 (+ 15us) server_negotiation.cc:407] Connection header received
0809 22:52:58.780023 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:52:58.780027 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:52:58.780113 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:52:58.780201 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:52:58.781196 (+ 995us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:52:58.781205 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:52:58.781209 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 22:52:58.781225 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:52:58.781257 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 22:52:58.781269 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:52:58.781364 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0809 22:52:58.781452 (+ 88us) server_negotiation.cc:299] Negotiation successful
0809 22:52:58.781497 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260809 22:52:58.782358 23747 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:52:58.834486 24011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:52:58.833931 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.129:43463 (local address 127.0.0.1:56620)
0809 22:52:58.834167 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0809 22:52:58.834293 (+ 126us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.113.129:43463: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
I20260809 22:52:58.835472 24011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:52:58.835105 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.129:43463 (local address 127.0.0.1:56622)
0809 22:52:58.835254 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0809 22:52:58.835336 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.113.129:43463: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260809 22:52:58.867969 15814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskItYPJR/build/asan/bin/kudu with pid 23557
I20260809 22:52:58.893121 15814 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskItYPJR/build/asan/bin/kudu pid 23771
I20260809 22:52:58.895834 24011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:52:58.895357 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.129:43463 (local address 127.0.0.1:56634)
0809 22:52:58.895567 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0809 22:52:58.895671 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.113.129:43463: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
I20260809 22:52:58.896134 24010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:52:58.893832 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34632 (local address 127.15.113.131:37445)
0809 22:52:58.894017 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0809 22:52:58.894021 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 22:52:58.894040 (+ 19us) server_negotiation.cc:407] Connection header received
0809 22:52:58.894193 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:52:58.894197 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:52:58.894281 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:52:58.894363 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:52:58.895200 (+ 837us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:52:58.895207 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:52:58.895211 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 22:52:58.895230 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:52:58.895270 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 22:52:58.895287 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:52:58.895412 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0809 22:52:58.895739 (+ 327us) server_negotiation.cc:299] Negotiation successful
0809 22:52:58.895800 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260809 22:52:58.896803 23979 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:52:58.896835 24011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:52:58.896517 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.129:43463 (local address 127.0.0.1:56646)
0809 22:52:58.896660 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0809 22:52:58.896725 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.113.129:43463: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260809 22:52:58.977241 24011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:52:58.976715 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.129:43463 (local address 127.0.0.1:56660)
0809 22:52:58.976961 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0809 22:52:58.977075 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.113.129:43463: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":79}
I20260809 22:52:58.978312 24011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:52:58.977938 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.129:43463 (local address 127.0.0.1:56676)
0809 22:52:58.978127 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0809 22:52:58.978198 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.113.129:43463: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":94}
I20260809 22:52:58.983896 15814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskItYPJR/build/asan/bin/kudu with pid 23771
I20260809 22:52:59.007647 15814 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskItYPJR/build/asan/bin/kudu pid 23203
I20260809 22:52:59.008767 23304 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:52:59.114912 15814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskItYPJR/build/asan/bin/kudu with pid 23203
2026-08-09T22:52:59Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (2686 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260809 22:52:59.162932 15814 mini_oidc.cc:152] Starting JWKS server
I20260809 22:52:59.165778 15814 webserver.cc:460] Webserver started at https://127.0.0.1:37705/ using document root <none> and password file <none>
I20260809 22:52:59.166082 15814 mini_oidc.cc:189] Starting OIDC Discovery server
I20260809 22:52:59.166699 15814 webserver.cc:460] Webserver started at http://127.0.0.1:39273/ using document root <none> and password file <none>
2026-08-09T22:52:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T22:52:59Z Disabled control of system clock
I20260809 22:52:59.198554 15814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskItYPJR/build/asan/bin/kudu
/tmp/dist-test-taskItYPJR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786315935617919-15814-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786315935617919-15814-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786315935617919-15814-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786315935617919-15814-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.113.190:46015
--webserver_interface=127.15.113.190
--webserver_port=0
--builtin_ntp_servers=127.15.113.148:32943
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.113.190:46015
--enable_jwt_token_auth=true
--jwks_url=https://localhost:37705/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786315935617919-15814-0/testchainca.cert with env {}
W20260809 22:52:59.517359 24070 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:52:59.517678 24070 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:52:59.517817 24070 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:52:59.524977 24070 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 22:52:59.525099 24070 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:52:59.525173 24070 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:52:59.525259 24070 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 22:52:59.525318 24070 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260809 22:52:59.525403 24070 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260809 22:52:59.525454 24070 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:37705/jwks.json
I20260809 22:52:59.535367 24070 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.113.148:32943
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786315935617919-15814-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786315935617919-15814-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.113.190:46015
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.113.190:46015
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:37705/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786315935617919-15814-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.113.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786315935617919-15814-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786315935617919-15814-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:52:59.536644 24070 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:52:59.537951 24070 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:52:59.549055 24079 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:52:59.549095 24080 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:52:59.549660 24082 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:52:59.549764 24070 server_base.cc:1034] running on GCE node
I20260809 22:52:59.550522 24070 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:52:59.551802 24070 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:52:59.552976 24070 hybrid_clock.cc:648] HybridClock initialized: now 1786315979552919 us; error 48 us; skew 500 ppm
I20260809 22:52:59.553299 24070 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 22:52:59.555424 24070 webserver.cc:460] Webserver started at http://127.15.113.190:44399/ using document root <none> and password file <none>
I20260809 22:52:59.556066 24070 fs_manager.cc:359] Metadata directory not provided
I20260809 22:52:59.556170 24070 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:52:59.556458 24070 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:52:59.559230 24070 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786315935617919-15814-0/minicluster-data/master-0/data/instance:
uuid: "441e88233fc04ca08d2c233594d19e9a"
format_stamp: "Formatted at 2026-08-09 22:52:59 on dist-test-slave-ncp9"
server_key: "3a6406dca1632f472576f1e93311ccba"
server_key_iv: "bf082fb7ddc00fd89b0bac41f2e6aac8"
server_key_version: "encryptionkey@0"
I20260809 22:52:59.559990 24070 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786315935617919-15814-0/minicluster-data/master-0/wal/instance:
uuid: "441e88233fc04ca08d2c233594d19e9a"
format_stamp: "Formatted at 2026-08-09 22:52:59 on dist-test-slave-ncp9"
server_key: "3a6406dca1632f472576f1e93311ccba"
server_key_iv: "bf082fb7ddc00fd89b0bac41f2e6aac8"
server_key_version: "encryptionkey@0"
I20260809 22:52:59.564774 24070 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260809 22:52:59.568013 24094 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:52:59.569257 24070 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 22:52:59.569428 24070 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786315935617919-15814-0/minicluster-data/master-0/data,/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786315935617919-15814-0/minicluster-data/master-0/wal
uuid: "441e88233fc04ca08d2c233594d19e9a"
format_stamp: "Formatted at 2026-08-09 22:52:59 on dist-test-slave-ncp9"
server_key: "3a6406dca1632f472576f1e93311ccba"
server_key_iv: "bf082fb7ddc00fd89b0bac41f2e6aac8"
server_key_version: "encryptionkey@0"
I20260809 22:52:59.569618 24070 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786315935617919-15814-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786315935617919-15814-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786315935617919-15814-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:52:59.585836 24070 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:52:59.586547 24070 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:52:59.586831 24070 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:52:59.603530 24070 rpc_server.cc:307] RPC server started. Bound to: 127.15.113.190:46015
I20260809 22:52:59.603556 24171 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.113.190:46015 every 8 connection(s)
I20260809 22:52:59.604717 24070 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786315935617919-15814-0/minicluster-data/master-0/data/info.pb
I20260809 22:52:59.605541 15814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskItYPJR/build/asan/bin/kudu as pid 24070
I20260809 22:52:59.605769 15814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786315935617919-15814-0/minicluster-data/master-0/wal/instance
I20260809 22:52:59.606564 15814 external_mini_cluster.cc:1428] Setting key 104e2cf68b49056d0f5cdbc3193be690
I20260809 22:52:59.609231 24172 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:52:59.619889 24176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:52:59.608396 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53232 (local address 127.15.113.190:46015)
0809 22:52:59.609376 (+ 980us) server_negotiation.cc:206] Beginning negotiation
0809 22:52:59.609386 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 22:52:59.609426 (+ 40us) server_negotiation.cc:407] Connection header received
0809 22:52:59.611106 (+ 1680us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:52:59.611127 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:52:59.611549 (+ 422us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:52:59.611879 (+ 330us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:52:59.612509 (+ 630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:52:59.613867 (+ 1358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:52:59.615710 (+ 1843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:52:59.616313 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:52:59.616433 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:52:59.617718 (+ 1285us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:52:59.617725 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:52:59.617737 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 22:52:59.618263 (+ 526us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:52:59.618334 (+ 71us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 22:52:59.618358 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:52:59.618447 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0809 22:52:59.618701 (+ 254us) server_negotiation.cc:299] Negotiation successful
0809 22:52:59.618992 (+ 291us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":573,"thread_start_us":356,"threads_started":1}
I20260809 22:52:59.621449 24172 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 441e88233fc04ca08d2c233594d19e9a: Bootstrap starting.
I20260809 22:52:59.623718 15814 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 22:52:59.624854 24172 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 441e88233fc04ca08d2c233594d19e9a: Neither blocks nor log segments found. Creating new log.
I20260809 22:52:59.625851 24172 log.cc:824] T 00000000000000000000000000000000 P 441e88233fc04ca08d2c233594d19e9a: Log is configured to *not* fsync() on all Append() calls
I20260809 22:52:59.628489 24172 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 441e88233fc04ca08d2c233594d19e9a: No bootstrap required, opened a new log
I20260809 22:52:59.635494 24172 raft_consensus.cc:348] T 00000000000000000000000000000000 P 441e88233fc04ca08d2c233594d19e9a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "441e88233fc04ca08d2c233594d19e9a" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 46015 } }
I20260809 22:52:59.635844 24172 raft_consensus.cc:374] T 00000000000000000000000000000000 P 441e88233fc04ca08d2c233594d19e9a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:52:59.635942 24172 raft_consensus.cc:729] T 00000000000000000000000000000000 P 441e88233fc04ca08d2c233594d19e9a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 441e88233fc04ca08d2c233594d19e9a, State: Initialized, Role: FOLLOWER
I20260809 22:52:59.636473 24172 consensus_queue.cc:260] T 00000000000000000000000000000000 P 441e88233fc04ca08d2c233594d19e9a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "441e88233fc04ca08d2c233594d19e9a" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 46015 } }
I20260809 22:52:59.636696 24172 raft_consensus.cc:388] T 00000000000000000000000000000000 P 441e88233fc04ca08d2c233594d19e9a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 22:52:59.636821 24172 raft_consensus.cc:482] T 00000000000000000000000000000000 P 441e88233fc04ca08d2c233594d19e9a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 22:52:59.636997 24172 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 441e88233fc04ca08d2c233594d19e9a [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:52:59.639035 24172 raft_consensus.cc:504] T 00000000000000000000000000000000 P 441e88233fc04ca08d2c233594d19e9a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "441e88233fc04ca08d2c233594d19e9a" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 46015 } }
I20260809 22:52:59.639508 24172 leader_election.cc:302] T 00000000000000000000000000000000 P 441e88233fc04ca08d2c233594d19e9a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 441e88233fc04ca08d2c233594d19e9a; no voters:
I20260809 22:52:59.640033 24172 leader_election.cc:288] T 00000000000000000000000000000000 P 441e88233fc04ca08d2c233594d19e9a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 22:52:59.640146 24180 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 441e88233fc04ca08d2c233594d19e9a [term 1 FOLLOWER]: Leader election won for term 1
I20260809 22:52:59.640784 24180 raft_consensus.cc:686] T 00000000000000000000000000000000 P 441e88233fc04ca08d2c233594d19e9a [term 1 LEADER]: Becoming Leader. State: Replica: 441e88233fc04ca08d2c233594d19e9a, State: Running, Role: LEADER
I20260809 22:52:59.641299 24180 consensus_queue.cc:237] T 00000000000000000000000000000000 P 441e88233fc04ca08d2c233594d19e9a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "441e88233fc04ca08d2c233594d19e9a" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 46015 } }
I20260809 22:52:59.642022 24172 sys_catalog.cc:564] T 00000000000000000000000000000000 P 441e88233fc04ca08d2c233594d19e9a [sys.catalog]: configured and running, proceeding with master startup.
I20260809 22:52:59.644205 24182 sys_catalog.cc:455] T 00000000000000000000000000000000 P 441e88233fc04ca08d2c233594d19e9a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 441e88233fc04ca08d2c233594d19e9a. Latest consensus state: current_term: 1 leader_uuid: "441e88233fc04ca08d2c233594d19e9a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "441e88233fc04ca08d2c233594d19e9a" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 46015 } } }
I20260809 22:52:59.644127 24181 sys_catalog.cc:455] T 00000000000000000000000000000000 P 441e88233fc04ca08d2c233594d19e9a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "441e88233fc04ca08d2c233594d19e9a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "441e88233fc04ca08d2c233594d19e9a" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 46015 } } }
I20260809 22:52:59.644533 24182 sys_catalog.cc:458] T 00000000000000000000000000000000 P 441e88233fc04ca08d2c233594d19e9a [sys.catalog]: This master's current role is: LEADER
I20260809 22:52:59.648759 24181 sys_catalog.cc:458] T 00000000000000000000000000000000 P 441e88233fc04ca08d2c233594d19e9a [sys.catalog]: This master's current role is: LEADER
I20260809 22:52:59.649463 24192 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 22:52:59.653748 24192 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 22:52:59.663328 24192 catalog_manager.cc:1347] Generated new cluster ID: 139f8680ade642f38d88a94b4f098e70
I20260809 22:52:59.663440 24192 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 22:52:59.691233 24192 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 22:52:59.693133 24192 catalog_manager.cc:1504] Loading token signing keys...
I20260809 22:52:59.713848 24192 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 441e88233fc04ca08d2c233594d19e9a: Generated new TSK 0
I20260809 22:52:59.714717 24192 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260809 22:52:59.756453 24235 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260809 22:52:59.756664 24235 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260809 22:52:59.758761 24176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:52:59.734051 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53238 (local address 127.15.113.190:46015)
0809 22:52:59.734298 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0809 22:52:59.734306 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 22:52:59.734861 (+ 555us) server_negotiation.cc:407] Connection header received
0809 22:52:59.735203 (+ 342us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:52:59.735210 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:52:59.735302 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:52:59.735463 (+ 161us) server_negotiation.cc:226] Negotiated authn=JWT
0809 22:52:59.735995 (+ 532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:52:59.737198 (+ 1203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:52:59.738661 (+ 1463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:52:59.739388 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:52:59.739518 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:52:59.740036 (+ 518us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0809 22:52:59.758076 (+ 18040us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37705/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0809 22:52:59.758446 (+ 370us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:53238: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37705/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":68}
W20260809 22:52:59.758899 24176 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:53238: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37705/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260809 22:52:59.760285 24229 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:52:59.734119 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:46015 (local address 127.0.0.1:53238)
0809 22:52:59.734758 (+ 639us) negotiation.cc:107] Waiting for socket to connect
0809 22:52:59.734774 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 22:52:59.734929 (+ 155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:52:59.735504 (+ 575us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:52:59.735507 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:52:59.735517 (+ 10us) client_negotiation.cc:189] Negotiated authn=JWT
0809 22:52:59.735835 (+ 318us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:52:59.735844 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:52:59.737348 (+ 1504us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:52:59.737353 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:52:59.738517 (+ 1164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:52:59.738532 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:52:59.739586 (+ 1054us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:52:59.739591 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:52:59.739714 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:52:59.739771 (+ 57us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0809 22:52:59.758342 (+ 18571us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37705/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0809 22:52:59.758600 (+ 258us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.15.113.190:46015: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37705/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":470,"thread_start_us":333,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.15.113.190:46015: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37705/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260809 22:52:59.762272 15814 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskItYPJR/build/asan/bin/kudu pid 24070
I20260809 22:52:59.763440 24166 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:52:59.857100 15814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskItYPJR/build/asan/bin/kudu with pid 24070
2026-08-09T22:52:59Z chronyd exiting
I20260809 22:52:59.914541 15814 test_util.cc:175] -----------------------------------------------
I20260809 22:52:59.914693 15814 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786315935617919-15814-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (756 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260809 22:52:59.918469 15814 mini_oidc.cc:152] Starting JWKS server
I20260809 22:52:59.920241 15814 webserver.cc:460] Webserver started at https://127.0.0.1:41973/ using document root <none> and password file <none>
I20260809 22:52:59.920444 15814 mini_oidc.cc:189] Starting OIDC Discovery server
I20260809 22:52:59.921128 15814 webserver.cc:460] Webserver started at http://127.0.0.1:33895/ using document root <none> and password file <none>
2026-08-09T22:52:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T22:52:59Z Disabled control of system clock
I20260809 22:52:59.952188 15814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskItYPJR/build/asan/bin/kudu
/tmp/dist-test-taskItYPJR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786315935617919-15814-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786315935617919-15814-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786315935617919-15814-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786315935617919-15814-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.113.190:37125
--webserver_interface=127.15.113.190
--webserver_port=0
--builtin_ntp_servers=127.15.113.148:33809
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.113.190:37125
--enable_jwt_token_auth=true
--jwks_url=https://localhost:41973/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786315935617919-15814-0/testchainca.cert with env {}
W20260809 22:53:00.270483 24244 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:53:00.270773 24244 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:53:00.270910 24244 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:53:00.277784 24244 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 22:53:00.277886 24244 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:53:00.277949 24244 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:53:00.278029 24244 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 22:53:00.278090 24244 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260809 22:53:00.278174 24244 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260809 22:53:00.278230 24244 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:41973/jwks.json
I20260809 22:53:00.287787 24244 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.113.148:33809
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786315935617919-15814-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786315935617919-15814-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.113.190:37125
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.113.190:37125
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:41973/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786315935617919-15814-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.113.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786315935617919-15814-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786315935617919-15814-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:53:00.288973 24244 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:53:00.290222 24244 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:53:00.301290 24251 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:53:00.301234 24252 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:53:00.301265 24263 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:53:00.302237 24244 server_base.cc:1034] running on GCE node
I20260809 22:53:00.302969 24244 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:53:00.304232 24244 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:53:00.305419 24244 hybrid_clock.cc:648] HybridClock initialized: now 1786315980305374 us; error 35 us; skew 500 ppm
I20260809 22:53:00.305756 24244 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 22:53:00.313108 24244 webserver.cc:460] Webserver started at http://127.15.113.190:34317/ using document root <none> and password file <none>
I20260809 22:53:00.313756 24244 fs_manager.cc:359] Metadata directory not provided
I20260809 22:53:00.313867 24244 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:53:00.314177 24244 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:53:00.316967 24244 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786315935617919-15814-0/minicluster-data/master-0/data/instance:
uuid: "37880433fc604a749a8eee8b7fc7b78f"
format_stamp: "Formatted at 2026-08-09 22:53:00 on dist-test-slave-ncp9"
server_key: "5acc724d079d01fe03848243fb04c46b"
server_key_iv: "b2eb19249eac65124c4f12c17b8c0999"
server_key_version: "encryptionkey@0"
I20260809 22:53:00.317729 24244 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786315935617919-15814-0/minicluster-data/master-0/wal/instance:
uuid: "37880433fc604a749a8eee8b7fc7b78f"
format_stamp: "Formatted at 2026-08-09 22:53:00 on dist-test-slave-ncp9"
server_key: "5acc724d079d01fe03848243fb04c46b"
server_key_iv: "b2eb19249eac65124c4f12c17b8c0999"
server_key_version: "encryptionkey@0"
I20260809 22:53:00.322506 24244 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260809 22:53:00.325703 24271 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:53:00.326893 24244 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.004s
I20260809 22:53:00.327057 24244 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786315935617919-15814-0/minicluster-data/master-0/data,/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786315935617919-15814-0/minicluster-data/master-0/wal
uuid: "37880433fc604a749a8eee8b7fc7b78f"
format_stamp: "Formatted at 2026-08-09 22:53:00 on dist-test-slave-ncp9"
server_key: "5acc724d079d01fe03848243fb04c46b"
server_key_iv: "b2eb19249eac65124c4f12c17b8c0999"
server_key_version: "encryptionkey@0"
I20260809 22:53:00.327241 24244 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786315935617919-15814-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786315935617919-15814-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786315935617919-15814-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:53:00.343274 24244 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:53:00.343978 24244 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:53:00.344244 24244 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:53:00.361018 24244 rpc_server.cc:307] RPC server started. Bound to: 127.15.113.190:37125
I20260809 22:53:00.361034 24357 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.113.190:37125 every 8 connection(s)
I20260809 22:53:00.362351 24244 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786315935617919-15814-0/minicluster-data/master-0/data/info.pb
I20260809 22:53:00.366009 24358 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:53:00.368916 15814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskItYPJR/build/asan/bin/kudu as pid 24244
I20260809 22:53:00.369112 15814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786315935617919-15814-0/minicluster-data/master-0/wal/instance
I20260809 22:53:00.369921 15814 external_mini_cluster.cc:1428] Setting key 70e658672db72bd429aea869d12eee41
I20260809 22:53:00.378039 24358 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 37880433fc604a749a8eee8b7fc7b78f: Bootstrap starting.
I20260809 22:53:00.381534 24358 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 37880433fc604a749a8eee8b7fc7b78f: Neither blocks nor log segments found. Creating new log.
I20260809 22:53:00.382651 24358 log.cc:824] T 00000000000000000000000000000000 P 37880433fc604a749a8eee8b7fc7b78f: Log is configured to *not* fsync() on all Append() calls
I20260809 22:53:00.382916 24366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:00.371780 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35058 (local address 127.15.113.190:37125)
0809 22:53:00.372684 (+ 904us) server_negotiation.cc:206] Beginning negotiation
0809 22:53:00.372694 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 22:53:00.372734 (+ 40us) server_negotiation.cc:407] Connection header received
0809 22:53:00.374341 (+ 1607us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:53:00.374367 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:53:00.374822 (+ 455us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:53:00.375132 (+ 310us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:53:00.375679 (+ 547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:00.376973 (+ 1294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:00.378729 (+ 1756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:00.379358 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:00.379519 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:00.380679 (+ 1160us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:53:00.380686 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:53:00.380697 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 22:53:00.381218 (+ 521us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:53:00.381303 (+ 85us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 22:53:00.381329 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:53:00.381408 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0809 22:53:00.381652 (+ 244us) server_negotiation.cc:299] Negotiation successful
0809 22:53:00.381955 (+ 303us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":541,"thread_start_us":336,"threads_started":1}
I20260809 22:53:00.386389 24358 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 37880433fc604a749a8eee8b7fc7b78f: No bootstrap required, opened a new log
I20260809 22:53:00.386691 15814 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 22:53:00.394132 24358 raft_consensus.cc:348] T 00000000000000000000000000000000 P 37880433fc604a749a8eee8b7fc7b78f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "37880433fc604a749a8eee8b7fc7b78f" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 37125 } }
I20260809 22:53:00.394457 24358 raft_consensus.cc:374] T 00000000000000000000000000000000 P 37880433fc604a749a8eee8b7fc7b78f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:53:00.394532 24358 raft_consensus.cc:729] T 00000000000000000000000000000000 P 37880433fc604a749a8eee8b7fc7b78f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 37880433fc604a749a8eee8b7fc7b78f, State: Initialized, Role: FOLLOWER
I20260809 22:53:00.394979 24358 consensus_queue.cc:260] T 00000000000000000000000000000000 P 37880433fc604a749a8eee8b7fc7b78f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "37880433fc604a749a8eee8b7fc7b78f" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 37125 } }
I20260809 22:53:00.395182 24358 raft_consensus.cc:388] T 00000000000000000000000000000000 P 37880433fc604a749a8eee8b7fc7b78f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 22:53:00.395274 24358 raft_consensus.cc:482] T 00000000000000000000000000000000 P 37880433fc604a749a8eee8b7fc7b78f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 22:53:00.395398 24358 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 37880433fc604a749a8eee8b7fc7b78f [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:53:00.397431 24358 raft_consensus.cc:504] T 00000000000000000000000000000000 P 37880433fc604a749a8eee8b7fc7b78f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "37880433fc604a749a8eee8b7fc7b78f" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 37125 } }
I20260809 22:53:00.397908 24358 leader_election.cc:302] T 00000000000000000000000000000000 P 37880433fc604a749a8eee8b7fc7b78f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 37880433fc604a749a8eee8b7fc7b78f; no voters:
I20260809 22:53:00.398417 24358 leader_election.cc:288] T 00000000000000000000000000000000 P 37880433fc604a749a8eee8b7fc7b78f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 22:53:00.398541 24370 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 37880433fc604a749a8eee8b7fc7b78f [term 1 FOLLOWER]: Leader election won for term 1
I20260809 22:53:00.399202 24370 raft_consensus.cc:686] T 00000000000000000000000000000000 P 37880433fc604a749a8eee8b7fc7b78f [term 1 LEADER]: Becoming Leader. State: Replica: 37880433fc604a749a8eee8b7fc7b78f, State: Running, Role: LEADER
I20260809 22:53:00.399714 24370 consensus_queue.cc:237] T 00000000000000000000000000000000 P 37880433fc604a749a8eee8b7fc7b78f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "37880433fc604a749a8eee8b7fc7b78f" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 37125 } }
I20260809 22:53:00.400334 24358 sys_catalog.cc:564] T 00000000000000000000000000000000 P 37880433fc604a749a8eee8b7fc7b78f [sys.catalog]: configured and running, proceeding with master startup.
I20260809 22:53:00.402376 24372 sys_catalog.cc:455] T 00000000000000000000000000000000 P 37880433fc604a749a8eee8b7fc7b78f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 37880433fc604a749a8eee8b7fc7b78f. Latest consensus state: current_term: 1 leader_uuid: "37880433fc604a749a8eee8b7fc7b78f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "37880433fc604a749a8eee8b7fc7b78f" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 37125 } } }
I20260809 22:53:00.402390 24371 sys_catalog.cc:455] T 00000000000000000000000000000000 P 37880433fc604a749a8eee8b7fc7b78f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "37880433fc604a749a8eee8b7fc7b78f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "37880433fc604a749a8eee8b7fc7b78f" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 37125 } } }
I20260809 22:53:00.402804 24371 sys_catalog.cc:458] T 00000000000000000000000000000000 P 37880433fc604a749a8eee8b7fc7b78f [sys.catalog]: This master's current role is: LEADER
I20260809 22:53:00.406354 24372 sys_catalog.cc:458] T 00000000000000000000000000000000 P 37880433fc604a749a8eee8b7fc7b78f [sys.catalog]: This master's current role is: LEADER
I20260809 22:53:00.407769 24377 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 22:53:00.412544 24377 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 22:53:00.420786 24377 catalog_manager.cc:1347] Generated new cluster ID: e0f13be717d349de88659cf674b8e299
I20260809 22:53:00.420907 24377 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 22:53:00.450652 24377 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 22:53:00.452060 24377 catalog_manager.cc:1504] Loading token signing keys...
I20260809 22:53:00.463241 24377 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 37880433fc604a749a8eee8b7fc7b78f: Generated new TSK 0
I20260809 22:53:00.464145 24377 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260809 22:53:00.506623 24403 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260809 22:53:00.506731 24403 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260809 22:53:00.507933 24366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:00.487659 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35064 (local address 127.15.113.190:37125)
0809 22:53:00.487849 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0809 22:53:00.487855 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:53:00.488251 (+ 396us) server_negotiation.cc:407] Connection header received
0809 22:53:00.488453 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:53:00.488459 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:53:00.488546 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:53:00.488712 (+ 166us) server_negotiation.cc:226] Negotiated authn=JWT
0809 22:53:00.489130 (+ 418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:00.490262 (+ 1132us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:00.491575 (+ 1313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:00.492251 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:00.492381 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:00.492807 (+ 426us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0809 22:53:00.507273 (+ 14466us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41973/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0809 22:53:00.507637 (+ 364us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:35064: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41973/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":55}
W20260809 22:53:00.507956 24402 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:53:00.487491 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:37125 (local address 127.0.0.1:35064)
0809 22:53:00.488150 (+ 659us) negotiation.cc:107] Waiting for socket to connect
0809 22:53:00.488169 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 22:53:00.488283 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:53:00.488708 (+ 425us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:53:00.488711 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:53:00.488720 (+ 9us) client_negotiation.cc:189] Negotiated authn=JWT
0809 22:53:00.488989 (+ 269us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:53:00.488999 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:53:00.490371 (+ 1372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:53:00.490374 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:53:00.491428 (+ 1054us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:53:00.491440 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:53:00.492388 (+ 948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:53:00.492392 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:53:00.492525 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:00.492556 (+ 31us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0809 22:53:00.507479 (+ 14923us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41973/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0809 22:53:00.507658 (+ 179us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.15.113.190:37125: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41973/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":516,"thread_start_us":381,"threads_started":1}
W20260809 22:53:00.508068 24366 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:35064: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41973/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
I20260809 22:53:00.509405 15814 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskItYPJR/build/asan/bin/kudu pid 24244
I20260809 22:53:00.510504 24347 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:53:00.604360 15814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskItYPJR/build/asan/bin/kudu with pid 24244
2026-08-09T22:53:00Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (747 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260809 22:53:00.665769 15814 mini_oidc.cc:152] Starting JWKS server
I20260809 22:53:00.667537 15814 webserver.cc:460] Webserver started at https://127.0.0.1:33007/ using document root <none> and password file <none>
I20260809 22:53:00.667753 15814 mini_oidc.cc:189] Starting OIDC Discovery server
I20260809 22:53:00.668323 15814 webserver.cc:460] Webserver started at http://127.0.0.1:35561/ using document root <none> and password file <none>
2026-08-09T22:53:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T22:53:00Z Disabled control of system clock
I20260809 22:53:00.698635 15814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskItYPJR/build/asan/bin/kudu
/tmp/dist-test-taskItYPJR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786315935617919-15814-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786315935617919-15814-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786315935617919-15814-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786315935617919-15814-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.113.190:33005
--webserver_interface=127.15.113.190
--webserver_port=0
--builtin_ntp_servers=127.15.113.148:37679
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.113.190:33005
--enable_jwt_token_auth=true
--jwks_url=https://localhost:33007/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260809 22:53:01.018824 24412 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:53:01.019098 24412 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:53:01.019222 24412 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:53:01.026226 24412 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 22:53:01.026335 24412 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:53:01.026397 24412 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:53:01.026474 24412 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 22:53:01.026541 24412 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260809 22:53:01.026623 24412 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260809 22:53:01.026670 24412 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:33007/jwks.json
I20260809 22:53:01.035437 24412 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.113.148:37679
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786315935617919-15814-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786315935617919-15814-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.113.190:33005
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.113.190:33005
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:33007/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786315935617919-15814-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.113.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786315935617919-15814-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:53:01.036744 24412 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:53:01.038518 24412 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:53:01.049639 24421 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:53:01.049685 24425 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:53:01.049597 24423 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:53:01.050422 24412 server_base.cc:1034] running on GCE node
I20260809 22:53:01.051180 24412 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:53:01.052457 24412 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:53:01.053678 24412 hybrid_clock.cc:648] HybridClock initialized: now 1786315981053611 us; error 34 us; skew 500 ppm
I20260809 22:53:01.054008 24412 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 22:53:01.056195 24412 webserver.cc:460] Webserver started at http://127.15.113.190:35247/ using document root <none> and password file <none>
I20260809 22:53:01.056838 24412 fs_manager.cc:359] Metadata directory not provided
I20260809 22:53:01.056967 24412 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:53:01.057269 24412 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:53:01.060005 24412 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786315935617919-15814-0/minicluster-data/master-0/data/instance:
uuid: "4910df493c8042a4b95494bd0def2bc0"
format_stamp: "Formatted at 2026-08-09 22:53:01 on dist-test-slave-ncp9"
server_key: "2696560059ec8bf463c854ed7ff4902e"
server_key_iv: "c0c081acab7297841d8346818017ab3a"
server_key_version: "encryptionkey@0"
I20260809 22:53:01.060762 24412 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786315935617919-15814-0/minicluster-data/master-0/wal/instance:
uuid: "4910df493c8042a4b95494bd0def2bc0"
format_stamp: "Formatted at 2026-08-09 22:53:01 on dist-test-slave-ncp9"
server_key: "2696560059ec8bf463c854ed7ff4902e"
server_key_iv: "c0c081acab7297841d8346818017ab3a"
server_key_version: "encryptionkey@0"
I20260809 22:53:01.065654 24412 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.000s
I20260809 22:53:01.068918 24438 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:53:01.070197 24412 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 22:53:01.070359 24412 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786315935617919-15814-0/minicluster-data/master-0/data,/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786315935617919-15814-0/minicluster-data/master-0/wal
uuid: "4910df493c8042a4b95494bd0def2bc0"
format_stamp: "Formatted at 2026-08-09 22:53:01 on dist-test-slave-ncp9"
server_key: "2696560059ec8bf463c854ed7ff4902e"
server_key_iv: "c0c081acab7297841d8346818017ab3a"
server_key_version: "encryptionkey@0"
I20260809 22:53:01.070552 24412 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786315935617919-15814-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786315935617919-15814-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786315935617919-15814-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:53:01.087801 24412 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:53:01.088550 24412 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:53:01.088838 24412 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:53:01.105710 24412 rpc_server.cc:307] RPC server started. Bound to: 127.15.113.190:33005
I20260809 22:53:01.105718 24522 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.113.190:33005 every 8 connection(s)
I20260809 22:53:01.106948 24412 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786315935617919-15814-0/minicluster-data/master-0/data/info.pb
I20260809 22:53:01.110476 24524 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:53:01.114809 15814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskItYPJR/build/asan/bin/kudu as pid 24412
I20260809 22:53:01.115000 15814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786315935617919-15814-0/minicluster-data/master-0/wal/instance
I20260809 22:53:01.115636 15814 external_mini_cluster.cc:1428] Setting key 0cbc7c2a73c6a1de49e27ec755deba04
I20260809 22:53:01.120752 24524 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4910df493c8042a4b95494bd0def2bc0: Bootstrap starting.
I20260809 22:53:01.123750 24524 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4910df493c8042a4b95494bd0def2bc0: Neither blocks nor log segments found. Creating new log.
I20260809 22:53:01.124711 24524 log.cc:824] T 00000000000000000000000000000000 P 4910df493c8042a4b95494bd0def2bc0: Log is configured to *not* fsync() on all Append() calls
I20260809 22:53:01.127523 24524 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4910df493c8042a4b95494bd0def2bc0: No bootstrap required, opened a new log
I20260809 22:53:01.128463 24527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:01.117430 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34382 (local address 127.15.113.190:33005)
0809 22:53:01.118417 (+ 987us) server_negotiation.cc:206] Beginning negotiation
0809 22:53:01.118430 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 22:53:01.118477 (+ 47us) server_negotiation.cc:407] Connection header received
0809 22:53:01.120132 (+ 1655us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:53:01.120154 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:53:01.120590 (+ 436us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:53:01.120964 (+ 374us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:53:01.121508 (+ 544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:01.122826 (+ 1318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:01.124180 (+ 1354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:01.124873 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:01.125074 (+ 201us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:01.126264 (+ 1190us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:53:01.126272 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:53:01.126285 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 22:53:01.126799 (+ 514us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:53:01.126873 (+ 74us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 22:53:01.126894 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:53:01.126989 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0809 22:53:01.127223 (+ 234us) server_negotiation.cc:299] Negotiation successful
0809 22:53:01.127507 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":576,"thread_start_us":363,"threads_started":1}
I20260809 22:53:01.132079 15814 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 22:53:01.135103 24524 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4910df493c8042a4b95494bd0def2bc0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4910df493c8042a4b95494bd0def2bc0" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 33005 } }
I20260809 22:53:01.135491 24524 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4910df493c8042a4b95494bd0def2bc0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:53:01.135593 24524 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4910df493c8042a4b95494bd0def2bc0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4910df493c8042a4b95494bd0def2bc0, State: Initialized, Role: FOLLOWER
I20260809 22:53:01.136109 24524 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4910df493c8042a4b95494bd0def2bc0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4910df493c8042a4b95494bd0def2bc0" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 33005 } }
I20260809 22:53:01.136320 24524 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4910df493c8042a4b95494bd0def2bc0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 22:53:01.136441 24524 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4910df493c8042a4b95494bd0def2bc0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 22:53:01.136600 24524 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4910df493c8042a4b95494bd0def2bc0 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:53:01.138850 24524 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4910df493c8042a4b95494bd0def2bc0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4910df493c8042a4b95494bd0def2bc0" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 33005 } }
I20260809 22:53:01.139338 24524 leader_election.cc:302] T 00000000000000000000000000000000 P 4910df493c8042a4b95494bd0def2bc0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4910df493c8042a4b95494bd0def2bc0; no voters:
I20260809 22:53:01.139876 24524 leader_election.cc:288] T 00000000000000000000000000000000 P 4910df493c8042a4b95494bd0def2bc0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 22:53:01.140009 24529 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4910df493c8042a4b95494bd0def2bc0 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 22:53:01.140678 24529 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4910df493c8042a4b95494bd0def2bc0 [term 1 LEADER]: Becoming Leader. State: Replica: 4910df493c8042a4b95494bd0def2bc0, State: Running, Role: LEADER
I20260809 22:53:01.141185 24529 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4910df493c8042a4b95494bd0def2bc0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4910df493c8042a4b95494bd0def2bc0" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 33005 } }
I20260809 22:53:01.141803 24524 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4910df493c8042a4b95494bd0def2bc0 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 22:53:01.143858 24531 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4910df493c8042a4b95494bd0def2bc0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4910df493c8042a4b95494bd0def2bc0. Latest consensus state: current_term: 1 leader_uuid: "4910df493c8042a4b95494bd0def2bc0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4910df493c8042a4b95494bd0def2bc0" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 33005 } } }
I20260809 22:53:01.144238 24531 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4910df493c8042a4b95494bd0def2bc0 [sys.catalog]: This master's current role is: LEADER
I20260809 22:53:01.144971 24530 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4910df493c8042a4b95494bd0def2bc0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4910df493c8042a4b95494bd0def2bc0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4910df493c8042a4b95494bd0def2bc0" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 33005 } } }
I20260809 22:53:01.145316 24530 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4910df493c8042a4b95494bd0def2bc0 [sys.catalog]: This master's current role is: LEADER
I20260809 22:53:01.145669 24538 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 22:53:01.151098 24538 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 22:53:01.159384 24538 catalog_manager.cc:1347] Generated new cluster ID: c913463e97484f39aa1f7749e0da2aac
I20260809 22:53:01.159489 24538 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 22:53:01.173070 24538 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 22:53:01.174710 24538 catalog_manager.cc:1504] Loading token signing keys...
I20260809 22:53:01.181855 24538 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4910df493c8042a4b95494bd0def2bc0: Generated new TSK 0
I20260809 22:53:01.182586 24538 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 22:53:01.209950 24527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:01.201858 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34386 (local address 127.15.113.190:33005)
0809 22:53:01.202047 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0809 22:53:01.202054 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:53:01.202451 (+ 397us) server_negotiation.cc:407] Connection header received
0809 22:53:01.202657 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:53:01.202662 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:53:01.202751 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:53:01.202905 (+ 154us) server_negotiation.cc:226] Negotiated authn=JWT
0809 22:53:01.203313 (+ 408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:01.204434 (+ 1121us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:01.205814 (+ 1380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:01.206514 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:01.206646 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:01.207085 (+ 439us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0809 22:53:01.209336 (+ 2251us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33007/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0809 22:53:01.209652 (+ 316us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:34386: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33007/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":59}
W20260809 22:53:01.210083 24527 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:34386: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33007/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260809 22:53:01.211585 24567 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:53:01.201716 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:33005 (local address 127.0.0.1:34386)
0809 22:53:01.202362 (+ 646us) negotiation.cc:107] Waiting for socket to connect
0809 22:53:01.202376 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 22:53:01.202491 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:53:01.202903 (+ 412us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:53:01.202907 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:53:01.202915 (+ 8us) client_negotiation.cc:189] Negotiated authn=JWT
0809 22:53:01.203179 (+ 264us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:53:01.203187 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:53:01.204571 (+ 1384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:53:01.204574 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:53:01.205672 (+ 1098us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:53:01.205687 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:53:01.206680 (+ 993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:53:01.206684 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:53:01.206809 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:01.206840 (+ 31us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0809 22:53:01.209526 (+ 2686us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33007/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0809 22:53:01.211175 (+ 1649us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.15.113.190:33005: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33007/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":525,"mutex_wait_us":1374,"thread_start_us":339,"threads_started":1}
I20260809 22:53:01.212553 15814 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskItYPJR/build/asan/bin/kudu pid 24412
I20260809 22:53:01.213837 24510 generic_service.cc:196] Checking for leaks (request via RPC)
E20260809 22:53:01.224184 24568 webserver.cc:533] Webserver: sslize failed:
E20260809 22:53:01.224303 24568 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260809 22:53:01.305349 15814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskItYPJR/build/asan/bin/kudu with pid 24412
2026-08-09T22:53:01Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (699 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-08-09T22:53:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T22:53:01Z Disabled control of system clock
I20260809 22:53:01.399416 15814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskItYPJR/build/asan/bin/kudu
/tmp/dist-test-taskItYPJR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786315935617919-15814-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786315935617919-15814-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786315935617919-15814-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786315935617919-15814-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.113.190:35685
--webserver_interface=127.15.113.190
--webserver_port=0
--builtin_ntp_servers=127.15.113.148:45805
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.113.190:35685
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786315935617919-15814-0/testchainca.cert with env {}
W20260809 22:53:01.720507 24578 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:53:01.720793 24578 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:53:01.720877 24578 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:53:01.727903 24578 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 22:53:01.727999 24578 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:53:01.728035 24578 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:53:01.728091 24578 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 22:53:01.728132 24578 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260809 22:53:01.728191 24578 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260809 22:53:01.728229 24578 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260809 22:53:01.737753 24578 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.113.148:45805
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786315935617919-15814-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786315935617919-15814-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.113.190:35685
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.113.190:35685
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786315935617919-15814-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.113.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786315935617919-15814-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786315935617919-15814-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:53:01.738925 24578 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:53:01.740156 24578 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:53:01.751397 24586 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:53:01.751423 24587 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:53:01.751446 24589 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:53:01.752126 24578 server_base.cc:1034] running on GCE node
I20260809 22:53:01.752831 24578 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:53:01.754138 24578 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:53:01.755338 24578 hybrid_clock.cc:648] HybridClock initialized: now 1786315981755286 us; error 40 us; skew 500 ppm
I20260809 22:53:01.755664 24578 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 22:53:01.757742 24578 webserver.cc:460] Webserver started at http://127.15.113.190:38023/ using document root <none> and password file <none>
I20260809 22:53:01.758371 24578 fs_manager.cc:359] Metadata directory not provided
I20260809 22:53:01.758483 24578 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:53:01.758795 24578 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:53:01.761528 24578 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786315935617919-15814-0/minicluster-data/master-0/data/instance:
uuid: "8565d80e530640659caf98c2089e4029"
format_stamp: "Formatted at 2026-08-09 22:53:01 on dist-test-slave-ncp9"
server_key: "3eb78a4e8d6fd5229bc707adee607bbb"
server_key_iv: "0d868cc7052d7de4d5238c7802b3a65f"
server_key_version: "encryptionkey@0"
I20260809 22:53:01.762305 24578 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786315935617919-15814-0/minicluster-data/master-0/wal/instance:
uuid: "8565d80e530640659caf98c2089e4029"
format_stamp: "Formatted at 2026-08-09 22:53:01 on dist-test-slave-ncp9"
server_key: "3eb78a4e8d6fd5229bc707adee607bbb"
server_key_iv: "0d868cc7052d7de4d5238c7802b3a65f"
server_key_version: "encryptionkey@0"
I20260809 22:53:01.767135 24578 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260809 22:53:01.770902 24600 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:53:01.772119 24578 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 22:53:01.772277 24578 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786315935617919-15814-0/minicluster-data/master-0/data,/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786315935617919-15814-0/minicluster-data/master-0/wal
uuid: "8565d80e530640659caf98c2089e4029"
format_stamp: "Formatted at 2026-08-09 22:53:01 on dist-test-slave-ncp9"
server_key: "3eb78a4e8d6fd5229bc707adee607bbb"
server_key_iv: "0d868cc7052d7de4d5238c7802b3a65f"
server_key_version: "encryptionkey@0"
I20260809 22:53:01.772465 24578 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786315935617919-15814-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786315935617919-15814-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786315935617919-15814-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:53:01.797067 24578 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:53:01.797853 24578 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:53:01.798118 24578 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:53:01.815186 24578 rpc_server.cc:307] RPC server started. Bound to: 127.15.113.190:35685
I20260809 22:53:01.815157 24673 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.113.190:35685 every 8 connection(s)
I20260809 22:53:01.816447 24578 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786315935617919-15814-0/minicluster-data/master-0/data/info.pb
I20260809 22:53:01.820029 24676 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:53:01.825613 15814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskItYPJR/build/asan/bin/kudu as pid 24578
I20260809 22:53:01.825793 15814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786315935617919-15814-0/minicluster-data/master-0/wal/instance
I20260809 22:53:01.826437 15814 external_mini_cluster.cc:1428] Setting key 149da064a745ff08b1ed2d87c44a5191
I20260809 22:53:01.830192 24676 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8565d80e530640659caf98c2089e4029: Bootstrap starting.
I20260809 22:53:01.833315 24676 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8565d80e530640659caf98c2089e4029: Neither blocks nor log segments found. Creating new log.
I20260809 22:53:01.834508 24676 log.cc:824] T 00000000000000000000000000000000 P 8565d80e530640659caf98c2089e4029: Log is configured to *not* fsync() on all Append() calls
I20260809 22:53:01.837522 24676 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8565d80e530640659caf98c2089e4029: No bootstrap required, opened a new log
I20260809 22:53:01.839702 24679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:01.828224 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53932 (local address 127.15.113.190:35685)
0809 22:53:01.829206 (+ 982us) server_negotiation.cc:206] Beginning negotiation
0809 22:53:01.829217 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 22:53:01.829259 (+ 42us) server_negotiation.cc:407] Connection header received
0809 22:53:01.830861 (+ 1602us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:53:01.830884 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:53:01.831309 (+ 425us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:53:01.831628 (+ 319us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:53:01.832207 (+ 579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:01.833506 (+ 1299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:01.835265 (+ 1759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:01.836116 (+ 851us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:01.836257 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:01.837443 (+ 1186us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:53:01.837450 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:53:01.837463 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 22:53:01.838094 (+ 631us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:53:01.838175 (+ 81us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 22:53:01.838202 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:53:01.838282 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0809 22:53:01.838494 (+ 212us) server_negotiation.cc:299] Negotiation successful
0809 22:53:01.838793 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":549,"thread_start_us":342,"threads_started":1}
I20260809 22:53:01.843380 15814 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 22:53:01.846498 24676 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8565d80e530640659caf98c2089e4029 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8565d80e530640659caf98c2089e4029" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 35685 } }
I20260809 22:53:01.846938 24676 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8565d80e530640659caf98c2089e4029 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:53:01.847047 24676 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8565d80e530640659caf98c2089e4029 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8565d80e530640659caf98c2089e4029, State: Initialized, Role: FOLLOWER
I20260809 22:53:01.847527 24676 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8565d80e530640659caf98c2089e4029 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8565d80e530640659caf98c2089e4029" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 35685 } }
I20260809 22:53:01.847738 24676 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8565d80e530640659caf98c2089e4029 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 22:53:01.847860 24676 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8565d80e530640659caf98c2089e4029 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 22:53:01.848001 24676 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8565d80e530640659caf98c2089e4029 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:53:01.850039 24676 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8565d80e530640659caf98c2089e4029 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8565d80e530640659caf98c2089e4029" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 35685 } }
I20260809 22:53:01.850524 24676 leader_election.cc:302] T 00000000000000000000000000000000 P 8565d80e530640659caf98c2089e4029 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8565d80e530640659caf98c2089e4029; no voters:
I20260809 22:53:01.851042 24676 leader_election.cc:288] T 00000000000000000000000000000000 P 8565d80e530640659caf98c2089e4029 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 22:53:01.851186 24682 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8565d80e530640659caf98c2089e4029 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 22:53:01.851815 24682 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8565d80e530640659caf98c2089e4029 [term 1 LEADER]: Becoming Leader. State: Replica: 8565d80e530640659caf98c2089e4029, State: Running, Role: LEADER
I20260809 22:53:01.852299 24682 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8565d80e530640659caf98c2089e4029 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8565d80e530640659caf98c2089e4029" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 35685 } }
I20260809 22:53:01.852845 24676 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8565d80e530640659caf98c2089e4029 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 22:53:01.855436 24686 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8565d80e530640659caf98c2089e4029 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8565d80e530640659caf98c2089e4029. Latest consensus state: current_term: 1 leader_uuid: "8565d80e530640659caf98c2089e4029" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8565d80e530640659caf98c2089e4029" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 35685 } } }
I20260809 22:53:01.855803 24686 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8565d80e530640659caf98c2089e4029 [sys.catalog]: This master's current role is: LEADER
I20260809 22:53:01.856540 24699 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 22:53:01.856516 24685 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8565d80e530640659caf98c2089e4029 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8565d80e530640659caf98c2089e4029" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8565d80e530640659caf98c2089e4029" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 35685 } } }
I20260809 22:53:01.856973 24685 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8565d80e530640659caf98c2089e4029 [sys.catalog]: This master's current role is: LEADER
I20260809 22:53:01.861594 24699 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 22:53:01.869426 24699 catalog_manager.cc:1347] Generated new cluster ID: acc80b5ae74841cb80c6f0a29f376f4c
I20260809 22:53:01.869513 24699 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 22:53:01.881014 24699 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 22:53:01.882715 24699 catalog_manager.cc:1504] Loading token signing keys...
I20260809 22:53:01.890841 24699 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8565d80e530640659caf98c2089e4029: Generated new TSK 0
I20260809 22:53:01.891541 24699 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 22:53:01.967763 24679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:01.907532 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53948 (local address 127.15.113.190:35685)
0809 22:53:01.907720 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0809 22:53:01.907727 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:53:01.908001 (+ 274us) server_negotiation.cc:407] Connection header received
0809 22:53:01.908184 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:53:01.908191 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:53:01.908286 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:53:01.908475 (+ 189us) server_negotiation.cc:226] Negotiated authn=JWT
0809 22:53:01.908858 (+ 383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:01.909980 (+ 1122us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:01.911407 (+ 1427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:01.912145 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:01.912287 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:01.912690 (+ 403us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0809 22:53:01.966929 (+ 54239us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0809 22:53:01.967389 (+ 460us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:53948: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":73}
W20260809 22:53:01.967814 24726 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:53:01.907364 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:35685 (local address 127.0.0.1:53948)
0809 22:53:01.907903 (+ 539us) negotiation.cc:107] Waiting for socket to connect
0809 22:53:01.907918 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 22:53:01.908030 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:53:01.908471 (+ 441us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:53:01.908474 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:53:01.908483 (+ 9us) client_negotiation.cc:189] Negotiated authn=JWT
0809 22:53:01.908716 (+ 233us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:53:01.908725 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:53:01.910138 (+ 1413us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:53:01.910142 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:53:01.911263 (+ 1121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:53:01.911276 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:53:01.912292 (+ 1016us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:53:01.912296 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:53:01.912414 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:01.912446 (+ 32us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0809 22:53:01.967239 (+ 54793us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0809 22:53:01.967493 (+ 254us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.15.113.190:35685: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":418,"thread_start_us":286,"threads_started":1}
W20260809 22:53:01.967922 24679 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:53948: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
I20260809 22:53:01.969290 15814 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskItYPJR/build/asan/bin/kudu pid 24578
I20260809 22:53:01.970566 24668 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:53:02.063925 15814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskItYPJR/build/asan/bin/kudu with pid 24578
2026-08-09T22:53:02Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (736 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (415 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (418 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315935617919-15814-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 22:53:02 dist-test-slave-ncp9 krb5kdc[24752](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 22:53:02 dist-test-slave-ncp9 krb5kdc[24752](info): set up 2 sockets
Aug 09 22:53:02 dist-test-slave-ncp9 krb5kdc[24752](info): commencing operation
krb5kdc: starting...
W20260809 22:53:04.073320 15814 mini_kdc.cc:121] Time spent starting KDC: real 1.135s user 0.006s sys 0.023s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315935617919-15814-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315935617919-15814-0/krb5kdc/test-user.keytab.
Aug 09 22:53:04 dist-test-slave-ncp9 krb5kdc[24752](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315984, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T22:53:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T22:53:04Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.113.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.113.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.113.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.190.keytab.
Entry for principal kudu/127.15.113.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.190.keytab.
I20260809 22:53:04.293292 15814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskItYPJR/build/asan/bin/kudu
/tmp/dist-test-taskItYPJR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315935617919-15814-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315935617919-15814-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315935617919-15814-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315935617919-15814-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.113.190:34007
--webserver_interface=127.15.113.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.190.keytab
--principal=kudu/127.15.113.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.113.148:34145
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.113.190:34007
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315935617919-15814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315935617919-15814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315935617919-15814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:53:04.606308 24775 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:53:04.606590 24775 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:53:04.606729 24775 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:53:04.613615 24775 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 22:53:04.613722 24775 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:53:04.613799 24775 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:53:04.613873 24775 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 22:53:04.613933 24775 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 22:53:04.623356 24775 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.113.148:34145
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315935617919-15814-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315935617919-15814-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.113.190:34007
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.190.keytab
--principal=kudu/127.15.113.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.113.190:34007
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315935617919-15814-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.113.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315935617919-15814-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:53:04.624568 24775 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:53:04.625881 24775 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:53:04.637748 24785 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:53:04.637789 24787 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:53:04.637814 24783 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:53:04.638530 24775 server_base.cc:1034] running on GCE node
I20260809 22:53:04.639293 24775 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:53:04.640609 24775 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:53:04.641810 24775 hybrid_clock.cc:648] HybridClock initialized: now 1786315984641749 us; error 44 us; skew 500 ppm
I20260809 22:53:04.642139 24775 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:53:04 dist-test-slave-ncp9 krb5kdc[24752](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315984, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.113.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:53:04.645043 24775 init.cc:377] Logged in from keytab as kudu/127.15.113.190@KRBTEST.COM (short username kudu)
I20260809 22:53:04.646443 24775 webserver.cc:460] Webserver started at http://127.15.113.190:33667/ using document root <none> and password file <none>
I20260809 22:53:04.647140 24775 fs_manager.cc:359] Metadata directory not provided
I20260809 22:53:04.647262 24775 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:53:04.647567 24775 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:53:04.650529 24775 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315935617919-15814-0/minicluster-data/master-0/data/instance:
uuid: "19a7e9f798794b57831de5747cfa8d52"
format_stamp: "Formatted at 2026-08-09 22:53:04 on dist-test-slave-ncp9"
server_key: "ff56bf8fe5388bef3d663f7a44527ffa"
server_key_iv: "ebccbcc241c5065fc9a37d858a63c649"
server_key_version: "encryptionkey@0"
I20260809 22:53:04.651309 24775 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315935617919-15814-0/minicluster-data/master-0/wal/instance:
uuid: "19a7e9f798794b57831de5747cfa8d52"
format_stamp: "Formatted at 2026-08-09 22:53:04 on dist-test-slave-ncp9"
server_key: "ff56bf8fe5388bef3d663f7a44527ffa"
server_key_iv: "ebccbcc241c5065fc9a37d858a63c649"
server_key_version: "encryptionkey@0"
I20260809 22:53:04.656220 24775 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260809 22:53:04.659581 24800 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:53:04.660805 24775 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 22:53:04.661002 24775 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315935617919-15814-0/minicluster-data/master-0/data,/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315935617919-15814-0/minicluster-data/master-0/wal
uuid: "19a7e9f798794b57831de5747cfa8d52"
format_stamp: "Formatted at 2026-08-09 22:53:04 on dist-test-slave-ncp9"
server_key: "ff56bf8fe5388bef3d663f7a44527ffa"
server_key_iv: "ebccbcc241c5065fc9a37d858a63c649"
server_key_version: "encryptionkey@0"
I20260809 22:53:04.661182 24775 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315935617919-15814-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315935617919-15814-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315935617919-15814-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:53:04.682580 24775 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:53:04.686156 24775 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:53:04.686398 24775 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:53:04.703076 24775 rpc_server.cc:307] RPC server started. Bound to: 127.15.113.190:34007
I20260809 22:53:04.703105 24894 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.113.190:34007 every 8 connection(s)
I20260809 22:53:04.704355 24775 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315935617919-15814-0/minicluster-data/master-0/data/info.pb
I20260809 22:53:04.707895 24896 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:53:04.710179 15814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskItYPJR/build/asan/bin/kudu as pid 24775
I20260809 22:53:04.710367 15814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315935617919-15814-0/minicluster-data/master-0/wal/instance
I20260809 22:53:04.710985 15814 external_mini_cluster.cc:1428] Setting key d57c95a5cf12a1c5174c15506e7855d0
I20260809 22:53:04.719296 24896 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 19a7e9f798794b57831de5747cfa8d52: Bootstrap starting.
I20260809 22:53:04.722909 24896 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 19a7e9f798794b57831de5747cfa8d52: Neither blocks nor log segments found. Creating new log.
I20260809 22:53:04.724160 24896 log.cc:824] T 00000000000000000000000000000000 P 19a7e9f798794b57831de5747cfa8d52: Log is configured to *not* fsync() on all Append() calls
Aug 09 22:53:04 dist-test-slave-ncp9 krb5kdc[24752](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315984, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.113.190@KRBTEST.COM
I20260809 22:53:04.726868 24896 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 19a7e9f798794b57831de5747cfa8d52: No bootstrap required, opened a new log
I20260809 22:53:04.733361 24899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:04.712966 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37102 (local address 127.15.113.190:34007)
0809 22:53:04.713728 (+ 762us) server_negotiation.cc:206] Beginning negotiation
0809 22:53:04.713739 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 22:53:04.713779 (+ 40us) server_negotiation.cc:407] Connection header received
0809 22:53:04.715418 (+ 1639us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:53:04.715441 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:53:04.715840 (+ 399us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:53:04.716152 (+ 312us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:53:04.717869 (+ 1717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:04.719235 (+ 1366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:04.721017 (+ 1782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:04.721678 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:04.721801 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:04.726387 (+ 4586us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:53:04.726394 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:53:04.726408 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:53:04.726445 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:53:04.729484 (+ 3039us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:53:04.730143 (+ 659us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:53:04.730149 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:53:04.730155 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:53:04.730312 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:53:04.730708 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:53:04.730712 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:53:04.730714 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:53:04.731230 (+ 516us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:53:04.731517 (+ 287us) server_negotiation.cc:1035] Waiting for connection context
0809 22:53:04.731959 (+ 442us) server_negotiation.cc:299] Negotiation successful
0809 22:53:04.732351 (+ 392us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":559,"thread_start_us":355,"threads_started":1}
I20260809 22:53:04.734413 24896 raft_consensus.cc:348] T 00000000000000000000000000000000 P 19a7e9f798794b57831de5747cfa8d52 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "19a7e9f798794b57831de5747cfa8d52" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 34007 } }
I20260809 22:53:04.734746 24896 raft_consensus.cc:374] T 00000000000000000000000000000000 P 19a7e9f798794b57831de5747cfa8d52 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:53:04.734835 24896 raft_consensus.cc:729] T 00000000000000000000000000000000 P 19a7e9f798794b57831de5747cfa8d52 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 19a7e9f798794b57831de5747cfa8d52, State: Initialized, Role: FOLLOWER
I20260809 22:53:04.735356 24896 consensus_queue.cc:260] T 00000000000000000000000000000000 P 19a7e9f798794b57831de5747cfa8d52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "19a7e9f798794b57831de5747cfa8d52" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 34007 } }
I20260809 22:53:04.735581 24896 raft_consensus.cc:388] T 00000000000000000000000000000000 P 19a7e9f798794b57831de5747cfa8d52 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 22:53:04.735710 24896 raft_consensus.cc:482] T 00000000000000000000000000000000 P 19a7e9f798794b57831de5747cfa8d52 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 22:53:04.735873 24896 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 19a7e9f798794b57831de5747cfa8d52 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:53:04.737986 24896 raft_consensus.cc:504] T 00000000000000000000000000000000 P 19a7e9f798794b57831de5747cfa8d52 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "19a7e9f798794b57831de5747cfa8d52" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 34007 } }
I20260809 22:53:04.738404 24896 leader_election.cc:302] T 00000000000000000000000000000000 P 19a7e9f798794b57831de5747cfa8d52 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 19a7e9f798794b57831de5747cfa8d52; no voters:
I20260809 22:53:04.738886 24896 leader_election.cc:288] T 00000000000000000000000000000000 P 19a7e9f798794b57831de5747cfa8d52 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 22:53:04.739001 24901 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 19a7e9f798794b57831de5747cfa8d52 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 22:53:04.739612 24901 raft_consensus.cc:686] T 00000000000000000000000000000000 P 19a7e9f798794b57831de5747cfa8d52 [term 1 LEADER]: Becoming Leader. State: Replica: 19a7e9f798794b57831de5747cfa8d52, State: Running, Role: LEADER
I20260809 22:53:04.740126 24901 consensus_queue.cc:237] T 00000000000000000000000000000000 P 19a7e9f798794b57831de5747cfa8d52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "19a7e9f798794b57831de5747cfa8d52" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 34007 } }
I20260809 22:53:04.740661 24896 sys_catalog.cc:564] T 00000000000000000000000000000000 P 19a7e9f798794b57831de5747cfa8d52 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 22:53:04.742652 24909 sys_catalog.cc:455] T 00000000000000000000000000000000 P 19a7e9f798794b57831de5747cfa8d52 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 19a7e9f798794b57831de5747cfa8d52. Latest consensus state: current_term: 1 leader_uuid: "19a7e9f798794b57831de5747cfa8d52" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "19a7e9f798794b57831de5747cfa8d52" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 34007 } } }
I20260809 22:53:04.742962 24909 sys_catalog.cc:458] T 00000000000000000000000000000000 P 19a7e9f798794b57831de5747cfa8d52 [sys.catalog]: This master's current role is: LEADER
I20260809 22:53:04.743660 24905 sys_catalog.cc:455] T 00000000000000000000000000000000 P 19a7e9f798794b57831de5747cfa8d52 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "19a7e9f798794b57831de5747cfa8d52" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "19a7e9f798794b57831de5747cfa8d52" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 34007 } } }
I20260809 22:53:04.743997 24905 sys_catalog.cc:458] T 00000000000000000000000000000000 P 19a7e9f798794b57831de5747cfa8d52 [sys.catalog]: This master's current role is: LEADER
I20260809 22:53:04.744429 24914 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 22:53:04.750692 24914 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 22:53:04.758744 24914 catalog_manager.cc:1347] Generated new cluster ID: 89a7f3ee0f9e4ae0a6e55f80b55c5f03
I20260809 22:53:04.758848 24914 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 22:53:04.798466 24914 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 22:53:04.800274 24914 catalog_manager.cc:1504] Loading token signing keys...
I20260809 22:53:04.808432 24914 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 19a7e9f798794b57831de5747cfa8d52: Generated new TSK 0
I20260809 22:53:04.809250 24914 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.113.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.113.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.113.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.129.keytab.
Entry for principal kudu/127.15.113.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.129.keytab.
I20260809 22:53:04.880551 15814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskItYPJR/build/asan/bin/kudu
/tmp/dist-test-taskItYPJR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315935617919-15814-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315935617919-15814-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315935617919-15814-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315935617919-15814-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.113.129:0
--local_ip_for_outbound_sockets=127.15.113.129
--webserver_interface=127.15.113.129
--webserver_port=0
--tserver_master_addrs=127.15.113.190:34007
--keytab_file=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.129.keytab
--principal=kudu/127.15.113.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.113.148:34145
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315935617919-15814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315935617919-15814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315935617919-15814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:53:05.275776 24943 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:53:05.276062 24943 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:53:05.276206 24943 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:53:05.283384 24943 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:53:05.283504 24943 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:53:05.283680 24943 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.113.129
I20260809 22:53:05.293143 24943 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.113.148:34145
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315935617919-15814-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315935617919-15814-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.129.keytab
--principal=kudu/127.15.113.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.113.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315935617919-15814-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.113.129
--webserver_port=0
--tserver_master_addrs=127.15.113.190:34007
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.113.129
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315935617919-15814-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:53:05.294379 24943 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:53:05.295666 24943 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:53:05.308396 24952 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:53:05.308444 24951 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:53:05.308632 24955 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:53:05.309526 24943 server_base.cc:1034] running on GCE node
I20260809 22:53:05.310061 24943 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:53:05.310920 24943 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:53:05.312112 24943 hybrid_clock.cc:648] HybridClock initialized: now 1786315985312053 us; error 53 us; skew 500 ppm
I20260809 22:53:05.312407 24943 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:53:05 dist-test-slave-ncp9 krb5kdc[24752](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315985, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.113.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:53:05.319520 24943 init.cc:377] Logged in from keytab as kudu/127.15.113.129@KRBTEST.COM (short username kudu)
I20260809 22:53:05.321017 24943 webserver.cc:460] Webserver started at http://127.15.113.129:37009/ using document root <none> and password file <none>
I20260809 22:53:05.321666 24943 fs_manager.cc:359] Metadata directory not provided
I20260809 22:53:05.321802 24943 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:53:05.322113 24943 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:53:05.325078 24943 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315935617919-15814-0/minicluster-data/ts-0/data/instance:
uuid: "67b8672120174fc49098e4d175e9f366"
format_stamp: "Formatted at 2026-08-09 22:53:05 on dist-test-slave-ncp9"
server_key: "57f69e93f00f3dd9b839512a66139cd1"
server_key_iv: "1a15f657ef7f48df75aebccbd15c1729"
server_key_version: "encryptionkey@0"
I20260809 22:53:05.325878 24943 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315935617919-15814-0/minicluster-data/ts-0/wal/instance:
uuid: "67b8672120174fc49098e4d175e9f366"
format_stamp: "Formatted at 2026-08-09 22:53:05 on dist-test-slave-ncp9"
server_key: "57f69e93f00f3dd9b839512a66139cd1"
server_key_iv: "1a15f657ef7f48df75aebccbd15c1729"
server_key_version: "encryptionkey@0"
I20260809 22:53:05.330955 24943 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 22:53:05.334369 24968 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:53:05.335572 24943 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 22:53:05.335739 24943 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315935617919-15814-0/minicluster-data/ts-0/data,/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315935617919-15814-0/minicluster-data/ts-0/wal
uuid: "67b8672120174fc49098e4d175e9f366"
format_stamp: "Formatted at 2026-08-09 22:53:05 on dist-test-slave-ncp9"
server_key: "57f69e93f00f3dd9b839512a66139cd1"
server_key_iv: "1a15f657ef7f48df75aebccbd15c1729"
server_key_version: "encryptionkey@0"
I20260809 22:53:05.335933 24943 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315935617919-15814-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315935617919-15814-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315935617919-15814-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:53:05.348726 24943 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:53:05.352396 24943 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:53:05.352692 24943 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:53:05.353615 24943 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:53:05.354935 24943 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:53:05.355020 24943 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:53:05.355146 24943 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:53:05.355206 24943 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:53:05.390268 24943 rpc_server.cc:307] RPC server started. Bound to: 127.15.113.129:34289
I20260809 22:53:05.390304 25165 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.113.129:34289 every 8 connection(s)
I20260809 22:53:05.391640 24943 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315935617919-15814-0/minicluster-data/ts-0/data/info.pb
I20260809 22:53:05.395323 15814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskItYPJR/build/asan/bin/kudu as pid 24943
I20260809 22:53:05.395525 15814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315935617919-15814-0/minicluster-data/ts-0/wal/instance
I20260809 22:53:05.396313 15814 external_mini_cluster.cc:1428] Setting key 7ddcb4b9da2517f392137b004c39b6fb
Aug 09 22:53:05 dist-test-slave-ncp9 krb5kdc[24752](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315985, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.113.129@KRBTEST.COM for kudu/127.15.113.190@KRBTEST.COM
I20260809 22:53:05.413132 25169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:05.394204 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.113.129:46087 (local address 127.15.113.190:34007)
0809 22:53:05.394684 (+ 480us) server_negotiation.cc:206] Beginning negotiation
0809 22:53:05.394689 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:53:05.395793 (+ 1104us) server_negotiation.cc:407] Connection header received
0809 22:53:05.397922 (+ 2129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:53:05.397928 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:53:05.398015 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:53:05.398123 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:53:05.400159 (+ 2036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:05.400862 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:05.403005 (+ 2143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:05.403389 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:05.403483 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:05.407382 (+ 3899us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:53:05.407387 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:53:05.407390 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:53:05.407422 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:53:05.410254 (+ 2832us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:53:05.410918 (+ 664us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:53:05.410924 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:53:05.410928 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:53:05.411055 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:53:05.411468 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:53:05.411474 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:53:05.411476 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:53:05.411741 (+ 265us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:53:05.411869 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0809 22:53:05.412537 (+ 668us) server_negotiation.cc:299] Negotiation successful
0809 22:53:05.412770 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":368,"thread_start_us":234,"threads_started":1}
I20260809 22:53:05.414011 25172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:05.394768 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:34007 (local address 127.15.113.129:46087)
0809 22:53:05.395630 (+ 862us) negotiation.cc:107] Waiting for socket to connect
0809 22:53:05.395684 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0809 22:53:05.397656 (+ 1972us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:53:05.398411 (+ 755us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:53:05.398423 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:53:05.399174 (+ 751us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:53:05.399985 (+ 811us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:53:05.400013 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:53:05.401041 (+ 1028us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:53:05.401047 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:53:05.402861 (+ 1814us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:53:05.402877 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:53:05.403618 (+ 741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:53:05.403627 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:53:05.403925 (+ 298us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:05.404634 (+ 709us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:53:05.404665 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:53:05.407176 (+ 2511us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:53:05.410377 (+ 3201us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:53:05.410386 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:53:05.410401 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:53:05.410803 (+ 402us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:53:05.411167 (+ 364us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:53:05.411172 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:53:05.411175 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:53:05.411355 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:53:05.411864 (+ 509us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:53:05.411871 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:53:05.412185 (+ 314us) client_negotiation.cc:769] Sending connection context
0809 22:53:05.412416 (+ 231us) client_negotiation.cc:240] Negotiation successful
0809 22:53:05.412920 (+ 504us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":565,"thread_start_us":335,"threads_started":1}
I20260809 22:53:05.415506 25166 heartbeater.cc:347] Connected to a master server at 127.15.113.190:34007
I20260809 22:53:05.415805 25166 heartbeater.cc:464] Registering TS with master...
I20260809 22:53:05.416478 25166 heartbeater.cc:511] Master 127.15.113.190:34007 requested a full tablet report, sending...
I20260809 22:53:05.418726 24828 ts_manager.cc:194] Registered new tserver with Master: 67b8672120174fc49098e4d175e9f366 (127.15.113.129:34289)
I20260809 22:53:05.420743 24828 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.113.129@KRBTEST.COM'} at 127.15.113.129:46087
WARNING: no policy specified for kudu/127.15.113.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.113.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.113.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.130.keytab.
Entry for principal kudu/127.15.113.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.130.keytab.
I20260809 22:53:05.449462 15814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskItYPJR/build/asan/bin/kudu
/tmp/dist-test-taskItYPJR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315935617919-15814-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315935617919-15814-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315935617919-15814-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315935617919-15814-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.113.130:0
--local_ip_for_outbound_sockets=127.15.113.130
--webserver_interface=127.15.113.130
--webserver_port=0
--tserver_master_addrs=127.15.113.190:34007
--keytab_file=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.130.keytab
--principal=kudu/127.15.113.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.113.148:34145
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315935617919-15814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315935617919-15814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315935617919-15814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:53:05.770579 25176 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:53:05.770829 25176 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:53:05.770926 25176 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:53:05.777652 25176 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:53:05.777741 25176 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:53:05.777853 25176 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.113.130
I20260809 22:53:05.787716 25176 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.113.148:34145
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315935617919-15814-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315935617919-15814-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.130.keytab
--principal=kudu/127.15.113.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.113.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315935617919-15814-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.113.130
--webserver_port=0
--tserver_master_addrs=127.15.113.190:34007
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.113.130
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315935617919-15814-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:53:05.788807 25176 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:53:05.790036 25176 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:53:05.802245 25188 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:53:05.802356 25187 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:53:05.802927 25191 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:53:05.803051 25176 server_base.cc:1034] running on GCE node
I20260809 22:53:05.803639 25176 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:53:05.804479 25176 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:53:05.805665 25176 hybrid_clock.cc:648] HybridClock initialized: now 1786315985805602 us; error 51 us; skew 500 ppm
I20260809 22:53:05.805940 25176 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:53:05 dist-test-slave-ncp9 krb5kdc[24752](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315985, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.113.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:53:05.809003 25176 init.cc:377] Logged in from keytab as kudu/127.15.113.130@KRBTEST.COM (short username kudu)
I20260809 22:53:05.810335 25176 webserver.cc:460] Webserver started at http://127.15.113.130:46335/ using document root <none> and password file <none>
I20260809 22:53:05.810971 25176 fs_manager.cc:359] Metadata directory not provided
I20260809 22:53:05.811089 25176 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:53:05.811373 25176 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:53:05.814201 25176 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315935617919-15814-0/minicluster-data/ts-1/data/instance:
uuid: "9910913325364670a8739c82d537a4f0"
format_stamp: "Formatted at 2026-08-09 22:53:05 on dist-test-slave-ncp9"
server_key: "d2ef452a0271f0510e27540361a37382"
server_key_iv: "639b4e9a6dbbb4bddb2107f79a6bdb62"
server_key_version: "encryptionkey@0"
I20260809 22:53:05.814965 25176 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315935617919-15814-0/minicluster-data/ts-1/wal/instance:
uuid: "9910913325364670a8739c82d537a4f0"
format_stamp: "Formatted at 2026-08-09 22:53:05 on dist-test-slave-ncp9"
server_key: "d2ef452a0271f0510e27540361a37382"
server_key_iv: "639b4e9a6dbbb4bddb2107f79a6bdb62"
server_key_version: "encryptionkey@0"
I20260809 22:53:05.819671 25176 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260809 22:53:05.822922 25201 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:53:05.824080 25176 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 22:53:05.824234 25176 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315935617919-15814-0/minicluster-data/ts-1/data,/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315935617919-15814-0/minicluster-data/ts-1/wal
uuid: "9910913325364670a8739c82d537a4f0"
format_stamp: "Formatted at 2026-08-09 22:53:05 on dist-test-slave-ncp9"
server_key: "d2ef452a0271f0510e27540361a37382"
server_key_iv: "639b4e9a6dbbb4bddb2107f79a6bdb62"
server_key_version: "encryptionkey@0"
I20260809 22:53:05.824405 25176 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315935617919-15814-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315935617919-15814-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315935617919-15814-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:53:05.843809 25176 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:53:05.847791 25176 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:53:05.848030 25176 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:53:05.848875 25176 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:53:05.850133 25176 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:53:05.850217 25176 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:53:05.850335 25176 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:53:05.850387 25176 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:53:05.880187 25176 rpc_server.cc:307] RPC server started. Bound to: 127.15.113.130:46119
I20260809 22:53:05.880254 25400 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.113.130:46119 every 8 connection(s)
I20260809 22:53:05.881421 25176 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315935617919-15814-0/minicluster-data/ts-1/data/info.pb
I20260809 22:53:05.888339 15814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskItYPJR/build/asan/bin/kudu as pid 25176
I20260809 22:53:05.888495 15814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315935617919-15814-0/minicluster-data/ts-1/wal/instance
I20260809 22:53:05.888947 15814 external_mini_cluster.cc:1428] Setting key f8c56f00285bda7b240d7e294b8959a8
Aug 09 22:53:05 dist-test-slave-ncp9 krb5kdc[24752](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315985, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.113.130@KRBTEST.COM for kudu/127.15.113.190@KRBTEST.COM
I20260809 22:53:05.902076 25169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:05.883952 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.113.130:48295 (local address 127.15.113.190:34007)
0809 22:53:05.884164 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0809 22:53:05.884170 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:53:05.885355 (+ 1185us) server_negotiation.cc:407] Connection header received
0809 22:53:05.887248 (+ 1893us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:53:05.887253 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:53:05.887321 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:53:05.887405 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:53:05.889066 (+ 1661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:05.889988 (+ 922us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:05.891986 (+ 1998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:05.892555 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:05.892673 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:05.896513 (+ 3840us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:53:05.896518 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:53:05.896522 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:53:05.896555 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:53:05.899042 (+ 2487us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:53:05.899799 (+ 757us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:53:05.899802 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:53:05.899805 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:53:05.899929 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:53:05.900371 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:53:05.900374 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:53:05.900376 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:53:05.900632 (+ 256us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:53:05.900750 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0809 22:53:05.901482 (+ 732us) server_negotiation.cc:299] Negotiation successful
0809 22:53:05.901739 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260809 22:53:05.902731 25405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:05.884424 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:34007 (local address 127.15.113.130:48295)
0809 22:53:05.885188 (+ 764us) negotiation.cc:107] Waiting for socket to connect
0809 22:53:05.885240 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0809 22:53:05.886998 (+ 1758us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:53:05.887639 (+ 641us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:53:05.887648 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:53:05.888202 (+ 554us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:53:05.888839 (+ 637us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:53:05.888859 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:53:05.890160 (+ 1301us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:53:05.890165 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:53:05.891835 (+ 1670us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:53:05.891849 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:53:05.892769 (+ 920us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:53:05.892775 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:53:05.893036 (+ 261us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:05.893731 (+ 695us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:53:05.893763 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:53:05.896268 (+ 2505us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:53:05.899182 (+ 2914us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:53:05.899190 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:53:05.899204 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:53:05.899699 (+ 495us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:53:05.900033 (+ 334us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:53:05.900037 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:53:05.900040 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:53:05.900265 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:53:05.900746 (+ 481us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:53:05.900754 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:53:05.901123 (+ 369us) client_negotiation.cc:769] Sending connection context
0809 22:53:05.901343 (+ 220us) client_negotiation.cc:240] Negotiation successful
0809 22:53:05.901696 (+ 353us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":523,"thread_start_us":286,"threads_started":1}
I20260809 22:53:05.904080 25401 heartbeater.cc:347] Connected to a master server at 127.15.113.190:34007
I20260809 22:53:05.904448 25401 heartbeater.cc:464] Registering TS with master...
I20260809 22:53:05.905280 25401 heartbeater.cc:511] Master 127.15.113.190:34007 requested a full tablet report, sending...
I20260809 22:53:05.906781 24828 ts_manager.cc:194] Registered new tserver with Master: 9910913325364670a8739c82d537a4f0 (127.15.113.130:46119)
I20260809 22:53:05.907546 24828 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.113.130@KRBTEST.COM'} at 127.15.113.130:48295
WARNING: no policy specified for kudu/127.15.113.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.113.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.113.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.131.keytab.
Entry for principal kudu/127.15.113.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.131.keytab.
I20260809 22:53:05.944293 15814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskItYPJR/build/asan/bin/kudu
/tmp/dist-test-taskItYPJR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315935617919-15814-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315935617919-15814-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315935617919-15814-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315935617919-15814-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.113.131:0
--local_ip_for_outbound_sockets=127.15.113.131
--webserver_interface=127.15.113.131
--webserver_port=0
--tserver_master_addrs=127.15.113.190:34007
--keytab_file=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.131.keytab
--principal=kudu/127.15.113.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.113.148:34145
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315935617919-15814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315935617919-15814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315935617919-15814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:53:06.264519 25410 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:53:06.264808 25410 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:53:06.264981 25410 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:53:06.271854 25410 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:53:06.271957 25410 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:53:06.272110 25410 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.113.131
I20260809 22:53:06.281132 25410 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.113.148:34145
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315935617919-15814-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315935617919-15814-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.131.keytab
--principal=kudu/127.15.113.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.113.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315935617919-15814-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.113.131
--webserver_port=0
--tserver_master_addrs=127.15.113.190:34007
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.113.131
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315935617919-15814-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:53:06.282282 25410 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:53:06.283499 25410 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:53:06.295667 25419 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:53:06.295701 25422 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:53:06.295732 25420 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:53:06.296380 25410 server_base.cc:1034] running on GCE node
I20260809 22:53:06.296972 25410 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:53:06.297792 25410 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:53:06.298982 25410 hybrid_clock.cc:648] HybridClock initialized: now 1786315986298936 us; error 29 us; skew 500 ppm
I20260809 22:53:06.299260 25410 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:53:06 dist-test-slave-ncp9 krb5kdc[24752](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315986, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.113.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:53:06.302229 25410 init.cc:377] Logged in from keytab as kudu/127.15.113.131@KRBTEST.COM (short username kudu)
I20260809 22:53:06.303537 25410 webserver.cc:460] Webserver started at http://127.15.113.131:38531/ using document root <none> and password file <none>
I20260809 22:53:06.304150 25410 fs_manager.cc:359] Metadata directory not provided
I20260809 22:53:06.304265 25410 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:53:06.304545 25410 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:53:06.307261 25410 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315935617919-15814-0/minicluster-data/ts-2/data/instance:
uuid: "708cfe26f1134c1bbaf5369b8a544661"
format_stamp: "Formatted at 2026-08-09 22:53:06 on dist-test-slave-ncp9"
server_key: "e33e6357f76803c132de26a35ce7ba33"
server_key_iv: "5f7ac69fcc3d4d07fa1a1fbcfca4c1de"
server_key_version: "encryptionkey@0"
I20260809 22:53:06.307991 25410 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315935617919-15814-0/minicluster-data/ts-2/wal/instance:
uuid: "708cfe26f1134c1bbaf5369b8a544661"
format_stamp: "Formatted at 2026-08-09 22:53:06 on dist-test-slave-ncp9"
server_key: "e33e6357f76803c132de26a35ce7ba33"
server_key_iv: "5f7ac69fcc3d4d07fa1a1fbcfca4c1de"
server_key_version: "encryptionkey@0"
I20260809 22:53:06.312709 25410 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.005s
I20260809 22:53:06.315896 25432 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:53:06.317135 25410 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 22:53:06.317276 25410 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315935617919-15814-0/minicluster-data/ts-2/data,/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315935617919-15814-0/minicluster-data/ts-2/wal
uuid: "708cfe26f1134c1bbaf5369b8a544661"
format_stamp: "Formatted at 2026-08-09 22:53:06 on dist-test-slave-ncp9"
server_key: "e33e6357f76803c132de26a35ce7ba33"
server_key_iv: "5f7ac69fcc3d4d07fa1a1fbcfca4c1de"
server_key_version: "encryptionkey@0"
I20260809 22:53:06.317458 25410 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315935617919-15814-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315935617919-15814-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315935617919-15814-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:53:06.330425 25410 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:53:06.334231 25410 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:53:06.334481 25410 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:53:06.335305 25410 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:53:06.336481 25410 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:53:06.336558 25410 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:53:06.336675 25410 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:53:06.336732 25410 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:53:06.366374 25410 rpc_server.cc:307] RPC server started. Bound to: 127.15.113.131:35825
I20260809 22:53:06.366436 25649 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.113.131:35825 every 8 connection(s)
I20260809 22:53:06.367573 25410 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315935617919-15814-0/minicluster-data/ts-2/data/info.pb
I20260809 22:53:06.375921 15814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskItYPJR/build/asan/bin/kudu as pid 25410
I20260809 22:53:06.376093 15814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315935617919-15814-0/minicluster-data/ts-2/wal/instance
I20260809 22:53:06.376703 15814 external_mini_cluster.cc:1428] Setting key c914497ddd4229eb18f40c8976cd9019
Aug 09 22:53:06 dist-test-slave-ncp9 krb5kdc[24752](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315986, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.113.131@KRBTEST.COM for kudu/127.15.113.190@KRBTEST.COM
I20260809 22:53:06.386838 25169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:06.370051 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.113.131:49481 (local address 127.15.113.190:34007)
0809 22:53:06.370240 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0809 22:53:06.370245 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:53:06.371366 (+ 1121us) server_negotiation.cc:407] Connection header received
0809 22:53:06.373059 (+ 1693us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:53:06.373063 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:53:06.373160 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:53:06.373245 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:53:06.374574 (+ 1329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:06.375308 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:06.377445 (+ 2137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:06.377897 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:06.378028 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:06.381420 (+ 3392us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:53:06.381425 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:53:06.381428 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:53:06.381459 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:53:06.383951 (+ 2492us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:53:06.384628 (+ 677us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:53:06.384633 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:53:06.384635 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:53:06.384764 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:53:06.385218 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:53:06.385222 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:53:06.385224 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:53:06.385480 (+ 256us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:53:06.385612 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0809 22:53:06.386271 (+ 659us) server_negotiation.cc:299] Negotiation successful
0809 22:53:06.386491 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260809 22:53:06.387672 25655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:06.370507 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:34007 (local address 127.15.113.131:49481)
0809 22:53:06.371206 (+ 699us) negotiation.cc:107] Waiting for socket to connect
0809 22:53:06.371242 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0809 22:53:06.372790 (+ 1548us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:53:06.373428 (+ 638us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:53:06.373435 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:53:06.373902 (+ 467us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:53:06.374448 (+ 546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:53:06.374466 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:53:06.375421 (+ 955us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:53:06.375425 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:53:06.377301 (+ 1876us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:53:06.377313 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:53:06.378115 (+ 802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:53:06.378123 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:53:06.378428 (+ 305us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:06.379062 (+ 634us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:53:06.379084 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:53:06.381217 (+ 2133us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:53:06.384088 (+ 2871us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:53:06.384097 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:53:06.384111 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:53:06.384501 (+ 390us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:53:06.384868 (+ 367us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:53:06.384873 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:53:06.384877 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:53:06.385094 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:53:06.385604 (+ 510us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:53:06.385611 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:53:06.385912 (+ 301us) client_negotiation.cc:769] Sending connection context
0809 22:53:06.386137 (+ 225us) client_negotiation.cc:240] Negotiation successful
0809 22:53:06.386605 (+ 468us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":466,"thread_start_us":276,"threads_started":1}
I20260809 22:53:06.388885 25650 heartbeater.cc:347] Connected to a master server at 127.15.113.190:34007
I20260809 22:53:06.389223 25650 heartbeater.cc:464] Registering TS with master...
I20260809 22:53:06.390064 25650 heartbeater.cc:511] Master 127.15.113.190:34007 requested a full tablet report, sending...
I20260809 22:53:06.393939 24828 ts_manager.cc:194] Registered new tserver with Master: 708cfe26f1134c1bbaf5369b8a544661 (127.15.113.131:35825)
I20260809 22:53:06.394956 24828 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.113.131@KRBTEST.COM'} at 127.15.113.131:49481
I20260809 22:53:06.405505 15814 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 22:53:06.423626 25166 heartbeater.cc:503] Master 127.15.113.190:34007 was elected leader, sending a full tablet report...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315935617919-15814-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315935617919-15814-0/krb5kdc/test-admin.keytab.
Aug 09 22:53:06 dist-test-slave-ncp9 krb5kdc[24752](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315986, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:53:06.435542 15814 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260809 22:53:06.440519 25662 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:53:06.437968 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48402 (local address 127.15.113.129:34289)
0809 22:53:06.438641 (+ 673us) server_negotiation.cc:206] Beginning negotiation
0809 22:53:06.438656 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0809 22:53:06.438680 (+ 24us) server_negotiation.cc:407] Connection header received
0809 22:53:06.438733 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:53:06.438739 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:53:06.438932 (+ 193us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:53:06.439066 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:53:06.440046 (+ 980us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48402: BlockingRecv error: recv got EOF from 127.0.0.1:48402 (error 108)
Metrics: {"server-negotiator.queue_time_us":506,"thread_start_us":342,"threads_started":1}
W20260809 22:53:06.440490 15814 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315935617919-15814-0/krb5kdc/krb5cc' of type FILE
Aug 09 22:53:06 dist-test-slave-ncp9 krb5kdc[24752](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315986, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:53:06.442003 15814 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 09 22:53:06 dist-test-slave-ncp9 krb5kdc[24752](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315986, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:53:06.443380 15814 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 09 22:53:06 dist-test-slave-ncp9 krb5kdc[24752](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315986, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.113.129@KRBTEST.COM
I20260809 22:53:06.457149 25662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:06.444692 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48410 (local address 127.15.113.129:34289)
0809 22:53:06.444872 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0809 22:53:06.444877 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:53:06.445153 (+ 276us) server_negotiation.cc:407] Connection header received
0809 22:53:06.445278 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:53:06.445281 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:53:06.445335 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:53:06.445429 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:53:06.446387 (+ 958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:06.447323 (+ 936us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:06.448443 (+ 1120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:06.449026 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:06.449129 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:06.452272 (+ 3143us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:53:06.452283 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:53:06.452296 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:53:06.452326 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:53:06.454977 (+ 2651us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:53:06.455503 (+ 526us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:53:06.455509 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:53:06.455511 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:53:06.455622 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:53:06.455888 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:53:06.455890 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:53:06.455892 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:53:06.456222 (+ 330us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:53:06.456433 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0809 22:53:06.456631 (+ 198us) server_negotiation.cc:299] Negotiation successful
0809 22:53:06.456865 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
W20260809 22:53:06.462301 25662 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:53:06.460608 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48418 (local address 127.15.113.129:34289)
0809 22:53:06.460769 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0809 22:53:06.460774 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:53:06.461096 (+ 322us) server_negotiation.cc:407] Connection header received
0809 22:53:06.461226 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:53:06.461231 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:53:06.461295 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:53:06.461370 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:53:06.462093 (+ 723us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48418: BlockingRecv error: recv got EOF from 127.0.0.1:48418 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
W20260809 22:53:06.462505 15814 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315935617919-15814-0/krb5kdc/krb5cc' of type FILE
Aug 09 22:53:06 dist-test-slave-ncp9 krb5kdc[24752](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315986, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:53:06.463853 15814 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 09 22:53:06 dist-test-slave-ncp9 krb5kdc[24752](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315986, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:53:06.465140 15814 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 09 22:53:06 dist-test-slave-ncp9 krb5kdc[24752](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315986, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.113.129@KRBTEST.COM
I20260809 22:53:06.477097 25662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:06.466326 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48424 (local address 127.15.113.129:34289)
0809 22:53:06.466487 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0809 22:53:06.466493 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:53:06.466620 (+ 127us) server_negotiation.cc:407] Connection header received
0809 22:53:06.466761 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:53:06.466765 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:53:06.466814 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:53:06.466884 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:53:06.467717 (+ 833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:06.468374 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:06.469591 (+ 1217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:06.470022 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:06.470126 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:06.473160 (+ 3034us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:53:06.473165 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:53:06.473168 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:53:06.473196 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:53:06.475275 (+ 2079us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:53:06.475789 (+ 514us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:53:06.475792 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:53:06.475795 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:53:06.475894 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:53:06.476159 (+ 265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:53:06.476162 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:53:06.476164 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:53:06.476383 (+ 219us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:53:06.476566 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0809 22:53:06.476736 (+ 170us) server_negotiation.cc:299] Negotiation successful
0809 22:53:06.476871 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
W20260809 22:53:06.480700 25662 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:53:06.479205 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48440 (local address 127.15.113.129:34289)
0809 22:53:06.479360 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0809 22:53:06.479365 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:53:06.479593 (+ 228us) server_negotiation.cc:407] Connection header received
0809 22:53:06.479712 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:53:06.479717 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:53:06.479781 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:53:06.479854 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:53:06.480491 (+ 637us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48440: BlockingRecv error: recv got EOF from 127.0.0.1:48440 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
W20260809 22:53:06.481176 15814 init.cc:244] Error while opening credential cache '/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315935617919-15814-0/krb5kdc/krb5cc' of type FILE: Runtime error: End of credential cache reached (filename: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315935617919-15814-0/krb5kdc/krb5cc)
Aug 09 22:53:06 dist-test-slave-ncp9 krb5kdc[24752](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315986, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:53:06.482470 15814 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 09 22:53:06 dist-test-slave-ncp9 krb5kdc[24752](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315986, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:53:06.483630 15814 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 09 22:53:06 dist-test-slave-ncp9 krb5kdc[24752](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315986, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.113.129@KRBTEST.COM
I20260809 22:53:06.496348 25662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:06.484707 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48454 (local address 127.15.113.129:34289)
0809 22:53:06.484871 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0809 22:53:06.484877 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:53:06.485121 (+ 244us) server_negotiation.cc:407] Connection header received
0809 22:53:06.485258 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:53:06.485263 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:53:06.485318 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:53:06.485390 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:53:06.486251 (+ 861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:06.486927 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:06.488078 (+ 1151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:06.488483 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:06.488584 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:06.491875 (+ 3291us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:53:06.491880 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:53:06.491883 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:53:06.491910 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:53:06.494089 (+ 2179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:53:06.494666 (+ 577us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:53:06.494670 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:53:06.494672 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:53:06.494811 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:53:06.495137 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:53:06.495141 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:53:06.495143 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:53:06.495381 (+ 238us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:53:06.495667 (+ 286us) server_negotiation.cc:1035] Waiting for connection context
0809 22:53:06.495898 (+ 231us) server_negotiation.cc:299] Negotiation successful
0809 22:53:06.496087 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260809 22:53:06.497721 15814 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskItYPJR/build/asan/bin/kudu pid 24943
I20260809 22:53:06.507836 25662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:06.498319 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48462 (local address 127.15.113.129:34289)
0809 22:53:06.498494 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0809 22:53:06.498499 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:53:06.498706 (+ 207us) server_negotiation.cc:407] Connection header received
0809 22:53:06.498809 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:53:06.498813 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:53:06.498863 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:53:06.498930 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:53:06.499661 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:06.500289 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:06.501471 (+ 1182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:06.501877 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:06.501975 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:06.503631 (+ 1656us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:53:06.503635 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:53:06.503639 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:53:06.503666 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:53:06.505846 (+ 2180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:53:06.506368 (+ 522us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:53:06.506372 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:53:06.506374 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:53:06.506490 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:53:06.506755 (+ 265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:53:06.506758 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:53:06.506760 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:53:06.506979 (+ 219us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:53:06.507163 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0809 22:53:06.507389 (+ 226us) server_negotiation.cc:299] Negotiation successful
0809 22:53:06.507539 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260809 22:53:06.508348 25161 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:53:06.589752 15814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskItYPJR/build/asan/bin/kudu with pid 24943
I20260809 22:53:06.611860 15814 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskItYPJR/build/asan/bin/kudu pid 25176
Aug 09 22:53:06 dist-test-slave-ncp9 krb5kdc[24752](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315986, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.113.130@KRBTEST.COM
I20260809 22:53:06.626500 25681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:06.612731 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44228 (local address 127.15.113.130:46119)
0809 22:53:06.613259 (+ 528us) server_negotiation.cc:206] Beginning negotiation
0809 22:53:06.613267 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 22:53:06.613292 (+ 25us) server_negotiation.cc:407] Connection header received
0809 22:53:06.613353 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:53:06.613359 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:53:06.613561 (+ 202us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:53:06.613724 (+ 163us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:53:06.614723 (+ 999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:06.615755 (+ 1032us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:06.617053 (+ 1298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:06.617613 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:06.617708 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:06.621345 (+ 3637us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:53:06.621357 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:53:06.621371 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:53:06.621413 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:53:06.624081 (+ 2668us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:53:06.624675 (+ 594us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:53:06.624681 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:53:06.624684 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:53:06.624801 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:53:06.625160 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:53:06.625163 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:53:06.625165 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:53:06.625498 (+ 333us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:53:06.625724 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0809 22:53:06.625948 (+ 224us) server_negotiation.cc:299] Negotiation successful
0809 22:53:06.626186 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":386,"thread_start_us":252,"threads_started":1}
I20260809 22:53:06.627808 25392 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:53:06.709741 15814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskItYPJR/build/asan/bin/kudu with pid 25176
I20260809 22:53:06.728529 15814 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskItYPJR/build/asan/bin/kudu pid 25410
Aug 09 22:53:06 dist-test-slave-ncp9 krb5kdc[24752](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315986, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.113.131@KRBTEST.COM
I20260809 22:53:06.743804 25684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:06.729563 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60900 (local address 127.15.113.131:35825)
0809 22:53:06.730211 (+ 648us) server_negotiation.cc:206] Beginning negotiation
0809 22:53:06.730218 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:53:06.730248 (+ 30us) server_negotiation.cc:407] Connection header received
0809 22:53:06.730310 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:53:06.730316 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:53:06.730536 (+ 220us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:53:06.730659 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:53:06.731631 (+ 972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:06.732660 (+ 1029us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:06.733923 (+ 1263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:06.734504 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:06.734616 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:06.738492 (+ 3876us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:53:06.738505 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:53:06.738520 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:53:06.738554 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:53:06.741121 (+ 2567us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:53:06.741777 (+ 656us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:53:06.741784 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:53:06.741787 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:53:06.741912 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:53:06.742348 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:53:06.742353 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:53:06.742356 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:53:06.742708 (+ 352us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:53:06.743020 (+ 312us) server_negotiation.cc:1035] Waiting for connection context
0809 22:53:06.743273 (+ 253us) server_negotiation.cc:299] Negotiation successful
0809 22:53:06.743526 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":504,"thread_start_us":342,"threads_started":1}
I20260809 22:53:06.745152 25643 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:53:06.826390 15814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskItYPJR/build/asan/bin/kudu with pid 25410
I20260809 22:53:06.849932 15814 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskItYPJR/build/asan/bin/kudu pid 24775
I20260809 22:53:06.851095 24888 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:53:06.953336 15814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskItYPJR/build/asan/bin/kudu with pid 24775
2026-08-09T22:53:06Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (4066 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315935617919-15814-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 22:53:07 dist-test-slave-ncp9 krb5kdc[25693](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 22:53:07 dist-test-slave-ncp9 krb5kdc[25693](info): set up 2 sockets
Aug 09 22:53:07 dist-test-slave-ncp9 krb5kdc[25693](info): commencing operation
krb5kdc: starting...
W20260809 22:53:08.170087 15814 mini_kdc.cc:121] Time spent starting KDC: real 1.166s user 0.001s sys 0.030s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315935617919-15814-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315935617919-15814-0/krb5kdc/test-user.keytab.
Aug 09 22:53:08 dist-test-slave-ncp9 krb5kdc[25693](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315988, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T22:53:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T22:53:08Z Disabled control of system clock
WARNING: no policy specified for oryx/127.15.113.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.15.113.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.15.113.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315935617919-15814-0/krb5kdc/oryx_127.15.113.190.keytab.
Entry for principal oryx/127.15.113.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315935617919-15814-0/krb5kdc/oryx_127.15.113.190.keytab.
I20260809 22:53:08.388134 15814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskItYPJR/build/asan/bin/kudu
/tmp/dist-test-taskItYPJR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315935617919-15814-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315935617919-15814-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315935617919-15814-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315935617919-15814-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.113.190:42225
--webserver_interface=127.15.113.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315935617919-15814-0/krb5kdc/oryx_127.15.113.190.keytab
--principal=oryx/127.15.113.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.113.148:37185
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.113.190:42225
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315935617919-15814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315935617919-15814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315935617919-15814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:53:08.707018 25717 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:53:08.707316 25717 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:53:08.707449 25717 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:53:08.715008 25717 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 22:53:08.715113 25717 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:53:08.715190 25717 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:53:08.715272 25717 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 22:53:08.715333 25717 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 22:53:08.725178 25717 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.113.148:37185
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315935617919-15814-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315935617919-15814-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.113.190:42225
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315935617919-15814-0/krb5kdc/oryx_127.15.113.190.keytab
--principal=oryx/127.15.113.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.113.190:42225
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315935617919-15814-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.113.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315935617919-15814-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:53:08.726635 25717 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:53:08.727948 25717 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:53:08.739215 25729 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:53:08.739251 25732 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:53:08.739684 25726 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:53:08.740149 25717 server_base.cc:1034] running on GCE node
I20260809 22:53:08.740871 25717 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:53:08.742252 25717 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:53:08.743436 25717 hybrid_clock.cc:648] HybridClock initialized: now 1786315988743380 us; error 44 us; skew 500 ppm
I20260809 22:53:08.743753 25717 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:53:08 dist-test-slave-ncp9 krb5kdc[25693](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315988, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.113.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:53:08.746685 25717 init.cc:377] Logged in from keytab as oryx/127.15.113.190@KRBTEST.COM (short username oryx)
I20260809 22:53:08.748013 25717 webserver.cc:460] Webserver started at http://127.15.113.190:37495/ using document root <none> and password file <none>
I20260809 22:53:08.748654 25717 fs_manager.cc:359] Metadata directory not provided
I20260809 22:53:08.748771 25717 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:53:08.749091 25717 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:53:08.752000 25717 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315935617919-15814-0/minicluster-data/master-0/data/instance:
uuid: "460ec5412d65461a811d766224ae4ab6"
format_stamp: "Formatted at 2026-08-09 22:53:08 on dist-test-slave-ncp9"
server_key: "cdb2686f97855959dfa4e7e728f0e97c"
server_key_iv: "0b32cdfdd3dde68f93bd623285bc4fd9"
server_key_version: "encryptionkey@0"
I20260809 22:53:08.752730 25717 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315935617919-15814-0/minicluster-data/master-0/wal/instance:
uuid: "460ec5412d65461a811d766224ae4ab6"
format_stamp: "Formatted at 2026-08-09 22:53:08 on dist-test-slave-ncp9"
server_key: "cdb2686f97855959dfa4e7e728f0e97c"
server_key_iv: "0b32cdfdd3dde68f93bd623285bc4fd9"
server_key_version: "encryptionkey@0"
I20260809 22:53:08.757611 25717 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260809 22:53:08.760721 25743 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:53:08.761937 25717 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260809 22:53:08.762095 25717 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315935617919-15814-0/minicluster-data/master-0/data,/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315935617919-15814-0/minicluster-data/master-0/wal
uuid: "460ec5412d65461a811d766224ae4ab6"
format_stamp: "Formatted at 2026-08-09 22:53:08 on dist-test-slave-ncp9"
server_key: "cdb2686f97855959dfa4e7e728f0e97c"
server_key_iv: "0b32cdfdd3dde68f93bd623285bc4fd9"
server_key_version: "encryptionkey@0"
I20260809 22:53:08.762284 25717 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315935617919-15814-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315935617919-15814-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315935617919-15814-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:53:08.802205 25717 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:53:08.805918 25717 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:53:08.806164 25717 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:53:08.823040 25717 rpc_server.cc:307] RPC server started. Bound to: 127.15.113.190:42225
I20260809 22:53:08.823081 25838 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.113.190:42225 every 8 connection(s)
I20260809 22:53:08.824330 25717 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315935617919-15814-0/minicluster-data/master-0/data/info.pb
I20260809 22:53:08.827083 15814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskItYPJR/build/asan/bin/kudu as pid 25717
I20260809 22:53:08.827306 15814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315935617919-15814-0/minicluster-data/master-0/wal/instance
I20260809 22:53:08.828145 15814 external_mini_cluster.cc:1428] Setting key e7984245bdaf7373f58ecdcd02dac356
I20260809 22:53:08.828547 25839 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:53:08.840637 25839 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 460ec5412d65461a811d766224ae4ab6: Bootstrap starting.
Aug 09 22:53:08 dist-test-slave-ncp9 krb5kdc[25693](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315988, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.15.113.190@KRBTEST.COM
I20260809 22:53:08.844094 25839 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 460ec5412d65461a811d766224ae4ab6: Neither blocks nor log segments found. Creating new log.
I20260809 22:53:08.845331 25839 log.cc:824] T 00000000000000000000000000000000 P 460ec5412d65461a811d766224ae4ab6: Log is configured to *not* fsync() on all Append() calls
I20260809 22:53:08.848157 25839 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 460ec5412d65461a811d766224ae4ab6: No bootstrap required, opened a new log
I20260809 22:53:08.849654 25842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:08.830233 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52172 (local address 127.15.113.190:42225)
0809 22:53:08.830979 (+ 746us) server_negotiation.cc:206] Beginning negotiation
0809 22:53:08.830990 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 22:53:08.831028 (+ 38us) server_negotiation.cc:407] Connection header received
0809 22:53:08.833183 (+ 2155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:53:08.833205 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:53:08.833565 (+ 360us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:53:08.833903 (+ 338us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:53:08.835340 (+ 1437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:08.836565 (+ 1225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:08.837794 (+ 1229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:08.838475 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:08.838647 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:08.842890 (+ 4243us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:53:08.842897 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:53:08.842911 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:53:08.842948 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:53:08.845993 (+ 3045us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:53:08.846636 (+ 643us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:53:08.846642 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:53:08.846655 (+ 13us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:53:08.846813 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:53:08.847156 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:53:08.847160 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:53:08.847162 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:53:08.847632 (+ 470us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:53:08.847920 (+ 288us) server_negotiation.cc:1035] Waiting for connection context
0809 22:53:08.848335 (+ 415us) server_negotiation.cc:299] Negotiation successful
0809 22:53:08.848717 (+ 382us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":557,"thread_start_us":331,"threads_started":1}
I20260809 22:53:08.856390 25839 raft_consensus.cc:348] T 00000000000000000000000000000000 P 460ec5412d65461a811d766224ae4ab6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "460ec5412d65461a811d766224ae4ab6" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 42225 } }
I20260809 22:53:08.856740 25839 raft_consensus.cc:374] T 00000000000000000000000000000000 P 460ec5412d65461a811d766224ae4ab6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:53:08.856837 25839 raft_consensus.cc:729] T 00000000000000000000000000000000 P 460ec5412d65461a811d766224ae4ab6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 460ec5412d65461a811d766224ae4ab6, State: Initialized, Role: FOLLOWER
I20260809 22:53:08.857445 25839 consensus_queue.cc:260] T 00000000000000000000000000000000 P 460ec5412d65461a811d766224ae4ab6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "460ec5412d65461a811d766224ae4ab6" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 42225 } }
I20260809 22:53:08.857674 25839 raft_consensus.cc:388] T 00000000000000000000000000000000 P 460ec5412d65461a811d766224ae4ab6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 22:53:08.857813 25839 raft_consensus.cc:482] T 00000000000000000000000000000000 P 460ec5412d65461a811d766224ae4ab6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 22:53:08.857988 25839 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 460ec5412d65461a811d766224ae4ab6 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:53:08.860373 25839 raft_consensus.cc:504] T 00000000000000000000000000000000 P 460ec5412d65461a811d766224ae4ab6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "460ec5412d65461a811d766224ae4ab6" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 42225 } }
I20260809 22:53:08.860939 25839 leader_election.cc:302] T 00000000000000000000000000000000 P 460ec5412d65461a811d766224ae4ab6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 460ec5412d65461a811d766224ae4ab6; no voters:
I20260809 22:53:08.861497 25839 leader_election.cc:288] T 00000000000000000000000000000000 P 460ec5412d65461a811d766224ae4ab6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 22:53:08.861619 25845 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 460ec5412d65461a811d766224ae4ab6 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 22:53:08.862311 25845 raft_consensus.cc:686] T 00000000000000000000000000000000 P 460ec5412d65461a811d766224ae4ab6 [term 1 LEADER]: Becoming Leader. State: Replica: 460ec5412d65461a811d766224ae4ab6, State: Running, Role: LEADER
I20260809 22:53:08.862784 25845 consensus_queue.cc:237] T 00000000000000000000000000000000 P 460ec5412d65461a811d766224ae4ab6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "460ec5412d65461a811d766224ae4ab6" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 42225 } }
I20260809 22:53:08.863570 25839 sys_catalog.cc:564] T 00000000000000000000000000000000 P 460ec5412d65461a811d766224ae4ab6 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 22:53:08.865473 25848 sys_catalog.cc:455] T 00000000000000000000000000000000 P 460ec5412d65461a811d766224ae4ab6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 460ec5412d65461a811d766224ae4ab6. Latest consensus state: current_term: 1 leader_uuid: "460ec5412d65461a811d766224ae4ab6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "460ec5412d65461a811d766224ae4ab6" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 42225 } } }
I20260809 22:53:08.866029 25848 sys_catalog.cc:458] T 00000000000000000000000000000000 P 460ec5412d65461a811d766224ae4ab6 [sys.catalog]: This master's current role is: LEADER
I20260809 22:53:08.866107 25847 sys_catalog.cc:455] T 00000000000000000000000000000000 P 460ec5412d65461a811d766224ae4ab6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "460ec5412d65461a811d766224ae4ab6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "460ec5412d65461a811d766224ae4ab6" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 42225 } } }
I20260809 22:53:08.866671 25847 sys_catalog.cc:458] T 00000000000000000000000000000000 P 460ec5412d65461a811d766224ae4ab6 [sys.catalog]: This master's current role is: LEADER
I20260809 22:53:08.869995 25855 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 22:53:08.875505 25855 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 22:53:08.883582 25855 catalog_manager.cc:1347] Generated new cluster ID: 2dcfd0b4c82745d78a39e01701586553
I20260809 22:53:08.883699 25855 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 22:53:08.899801 25855 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 22:53:08.900985 25855 catalog_manager.cc:1504] Loading token signing keys...
I20260809 22:53:08.911410 25855 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 460ec5412d65461a811d766224ae4ab6: Generated new TSK 0
I20260809 22:53:08.912242 25855 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.15.113.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.15.113.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.15.113.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315935617919-15814-0/krb5kdc/oryx_127.15.113.129.keytab.
Entry for principal oryx/127.15.113.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315935617919-15814-0/krb5kdc/oryx_127.15.113.129.keytab.
I20260809 22:53:08.988515 15814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskItYPJR/build/asan/bin/kudu
/tmp/dist-test-taskItYPJR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315935617919-15814-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315935617919-15814-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315935617919-15814-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315935617919-15814-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.113.129:0
--local_ip_for_outbound_sockets=127.15.113.129
--webserver_interface=127.15.113.129
--webserver_port=0
--tserver_master_addrs=127.15.113.190:42225
--keytab_file=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315935617919-15814-0/krb5kdc/oryx_127.15.113.129.keytab
--principal=oryx/127.15.113.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.113.148:37185
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315935617919-15814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315935617919-15814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315935617919-15814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:53:09.313417 25881 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:53:09.313679 25881 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:53:09.313804 25881 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:53:09.320772 25881 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:53:09.320869 25881 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:53:09.321034 25881 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.113.129
I20260809 22:53:09.330205 25881 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.113.148:37185
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315935617919-15814-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315935617919-15814-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315935617919-15814-0/krb5kdc/oryx_127.15.113.129.keytab
--principal=oryx/127.15.113.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.113.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315935617919-15814-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.113.129
--webserver_port=0
--tserver_master_addrs=127.15.113.190:42225
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.113.129
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315935617919-15814-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:53:09.331326 25881 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:53:09.332572 25881 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:53:09.344969 25890 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:53:09.345220 25893 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:53:09.345227 25897 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:53:09.346576 25881 server_base.cc:1034] running on GCE node
I20260809 22:53:09.347168 25881 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:53:09.348059 25881 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:53:09.349246 25881 hybrid_clock.cc:648] HybridClock initialized: now 1786315989349206 us; error 36 us; skew 500 ppm
I20260809 22:53:09.349521 25881 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:53:09 dist-test-slave-ncp9 krb5kdc[25693](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315989, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.113.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:53:09.352514 25881 init.cc:377] Logged in from keytab as oryx/127.15.113.129@KRBTEST.COM (short username oryx)
I20260809 22:53:09.353801 25881 webserver.cc:460] Webserver started at http://127.15.113.129:42149/ using document root <none> and password file <none>
I20260809 22:53:09.354450 25881 fs_manager.cc:359] Metadata directory not provided
I20260809 22:53:09.354564 25881 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:53:09.354838 25881 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:53:09.357609 25881 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315935617919-15814-0/minicluster-data/ts-0/data/instance:
uuid: "b8e298532adf4449955b1b5dc49552c2"
format_stamp: "Formatted at 2026-08-09 22:53:09 on dist-test-slave-ncp9"
server_key: "889049b1e44c67a336a386c60c7a0163"
server_key_iv: "55b58844f827b812997ddaeb0c80a78a"
server_key_version: "encryptionkey@0"
I20260809 22:53:09.358345 25881 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315935617919-15814-0/minicluster-data/ts-0/wal/instance:
uuid: "b8e298532adf4449955b1b5dc49552c2"
format_stamp: "Formatted at 2026-08-09 22:53:09 on dist-test-slave-ncp9"
server_key: "889049b1e44c67a336a386c60c7a0163"
server_key_iv: "55b58844f827b812997ddaeb0c80a78a"
server_key_version: "encryptionkey@0"
I20260809 22:53:09.363054 25881 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260809 22:53:09.366344 25904 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:53:09.367730 25881 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 22:53:09.367874 25881 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315935617919-15814-0/minicluster-data/ts-0/data,/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315935617919-15814-0/minicluster-data/ts-0/wal
uuid: "b8e298532adf4449955b1b5dc49552c2"
format_stamp: "Formatted at 2026-08-09 22:53:09 on dist-test-slave-ncp9"
server_key: "889049b1e44c67a336a386c60c7a0163"
server_key_iv: "55b58844f827b812997ddaeb0c80a78a"
server_key_version: "encryptionkey@0"
I20260809 22:53:09.368055 25881 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315935617919-15814-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315935617919-15814-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315935617919-15814-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:53:09.382628 25881 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:53:09.386025 25881 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:53:09.386281 25881 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:53:09.388785 25881 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:53:09.390054 25881 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:53:09.390139 25881 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:53:09.390262 25881 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:53:09.390318 25881 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:53:09.420321 25881 rpc_server.cc:307] RPC server started. Bound to: 127.15.113.129:38043
I20260809 22:53:09.420372 26092 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.113.129:38043 every 8 connection(s)
I20260809 22:53:09.421545 25881 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315935617919-15814-0/minicluster-data/ts-0/data/info.pb
I20260809 22:53:09.430493 15814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskItYPJR/build/asan/bin/kudu as pid 25881
I20260809 22:53:09.430675 15814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315935617919-15814-0/minicluster-data/ts-0/wal/instance
I20260809 22:53:09.431303 15814 external_mini_cluster.cc:1428] Setting key a2ba639bce664d891c89acec26502b49
Aug 09 22:53:09 dist-test-slave-ncp9 krb5kdc[25693](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315989, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.113.129@KRBTEST.COM for oryx/127.15.113.190@KRBTEST.COM
I20260809 22:53:09.440860 26095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:09.423931 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.113.129:38849 (local address 127.15.113.190:42225)
0809 22:53:09.424365 (+ 434us) server_negotiation.cc:206] Beginning negotiation
0809 22:53:09.424369 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 22:53:09.425340 (+ 971us) server_negotiation.cc:407] Connection header received
0809 22:53:09.427209 (+ 1869us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:53:09.427216 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:53:09.427297 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:53:09.427395 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:53:09.428834 (+ 1439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:09.429600 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:09.431222 (+ 1622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:09.431654 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:09.431773 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:09.435532 (+ 3759us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:53:09.435537 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:53:09.435541 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:53:09.435571 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:53:09.438001 (+ 2430us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:53:09.438786 (+ 785us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:53:09.438790 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:53:09.438792 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:53:09.438911 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:53:09.439343 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:53:09.439346 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:53:09.439348 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:53:09.439588 (+ 240us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:53:09.439695 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0809 22:53:09.440452 (+ 757us) server_negotiation.cc:299] Negotiation successful
0809 22:53:09.440614 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":322,"thread_start_us":211,"threads_started":1}
I20260809 22:53:09.442101 26098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:09.424455 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:42225 (local address 127.15.113.129:38849)
0809 22:53:09.425188 (+ 733us) negotiation.cc:107] Waiting for socket to connect
0809 22:53:09.425226 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0809 22:53:09.426939 (+ 1713us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:53:09.427646 (+ 707us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:53:09.427655 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:53:09.428123 (+ 468us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:53:09.428662 (+ 539us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:53:09.428681 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:53:09.429774 (+ 1093us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:53:09.429778 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:53:09.431073 (+ 1295us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:53:09.431086 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:53:09.431874 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:53:09.431881 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:53:09.432172 (+ 291us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:09.432828 (+ 656us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:53:09.432859 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:53:09.435318 (+ 2459us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:53:09.438141 (+ 2823us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:53:09.438149 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:53:09.438163 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:53:09.438673 (+ 510us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:53:09.439009 (+ 336us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:53:09.439013 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:53:09.439016 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:53:09.439256 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:53:09.439702 (+ 446us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:53:09.439709 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:53:09.440093 (+ 384us) client_negotiation.cc:769] Sending connection context
0809 22:53:09.440382 (+ 289us) client_negotiation.cc:240] Negotiation successful
0809 22:53:09.440845 (+ 463us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":491,"thread_start_us":269,"threads_started":1}
I20260809 22:53:09.443679 26093 heartbeater.cc:347] Connected to a master server at 127.15.113.190:42225
I20260809 22:53:09.443980 26093 heartbeater.cc:464] Registering TS with master...
I20260809 22:53:09.444830 26093 heartbeater.cc:511] Master 127.15.113.190:42225 requested a full tablet report, sending...
I20260809 22:53:09.447290 25770 ts_manager.cc:194] Registered new tserver with Master: b8e298532adf4449955b1b5dc49552c2 (127.15.113.129:38043)
I20260809 22:53:09.449275 25770 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.15.113.129@KRBTEST.COM'} at 127.15.113.129:38849
WARNING: no policy specified for oryx/127.15.113.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.15.113.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.15.113.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315935617919-15814-0/krb5kdc/oryx_127.15.113.130.keytab.
Entry for principal oryx/127.15.113.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315935617919-15814-0/krb5kdc/oryx_127.15.113.130.keytab.
I20260809 22:53:09.485605 15814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskItYPJR/build/asan/bin/kudu
/tmp/dist-test-taskItYPJR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315935617919-15814-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315935617919-15814-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315935617919-15814-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315935617919-15814-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.113.130:0
--local_ip_for_outbound_sockets=127.15.113.130
--webserver_interface=127.15.113.130
--webserver_port=0
--tserver_master_addrs=127.15.113.190:42225
--keytab_file=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315935617919-15814-0/krb5kdc/oryx_127.15.113.130.keytab
--principal=oryx/127.15.113.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.113.148:37185
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315935617919-15814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315935617919-15814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315935617919-15814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:53:09.815852 26101 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:53:09.816148 26101 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:53:09.816304 26101 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:53:09.824019 26101 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:53:09.824146 26101 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:53:09.824316 26101 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.113.130
I20260809 22:53:09.833891 26101 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.113.148:37185
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315935617919-15814-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315935617919-15814-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315935617919-15814-0/krb5kdc/oryx_127.15.113.130.keytab
--principal=oryx/127.15.113.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.113.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315935617919-15814-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.113.130
--webserver_port=0
--tserver_master_addrs=127.15.113.190:42225
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.113.130
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315935617919-15814-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:53:09.835043 26101 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:53:09.836351 26101 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:53:09.849290 26112 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:53:09.849321 26110 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:53:09.849848 26115 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:53:09.850093 26101 server_base.cc:1034] running on GCE node
I20260809 22:53:09.850688 26101 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:53:09.851567 26101 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:53:09.852770 26101 hybrid_clock.cc:648] HybridClock initialized: now 1786315989852717 us; error 38 us; skew 500 ppm
I20260809 22:53:09.853107 26101 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:53:09 dist-test-slave-ncp9 krb5kdc[25693](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315989, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.113.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:53:09.856247 26101 init.cc:377] Logged in from keytab as oryx/127.15.113.130@KRBTEST.COM (short username oryx)
I20260809 22:53:09.857739 26101 webserver.cc:460] Webserver started at http://127.15.113.130:34741/ using document root <none> and password file <none>
I20260809 22:53:09.858394 26101 fs_manager.cc:359] Metadata directory not provided
I20260809 22:53:09.858525 26101 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:53:09.858824 26101 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:53:09.861815 26101 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315935617919-15814-0/minicluster-data/ts-1/data/instance:
uuid: "200f1dd145df4993b427e20748be2114"
format_stamp: "Formatted at 2026-08-09 22:53:09 on dist-test-slave-ncp9"
server_key: "ed2210da7cb7d050dcc4392744efa732"
server_key_iv: "835280f3abcd0799fa0bf6da6eb0daa3"
server_key_version: "encryptionkey@0"
I20260809 22:53:09.862617 26101 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315935617919-15814-0/minicluster-data/ts-1/wal/instance:
uuid: "200f1dd145df4993b427e20748be2114"
format_stamp: "Formatted at 2026-08-09 22:53:09 on dist-test-slave-ncp9"
server_key: "ed2210da7cb7d050dcc4392744efa732"
server_key_iv: "835280f3abcd0799fa0bf6da6eb0daa3"
server_key_version: "encryptionkey@0"
I20260809 22:53:09.867784 26101 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260809 22:53:09.871347 26132 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:53:09.872656 26101 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 22:53:09.872818 26101 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315935617919-15814-0/minicluster-data/ts-1/data,/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315935617919-15814-0/minicluster-data/ts-1/wal
uuid: "200f1dd145df4993b427e20748be2114"
format_stamp: "Formatted at 2026-08-09 22:53:09 on dist-test-slave-ncp9"
server_key: "ed2210da7cb7d050dcc4392744efa732"
server_key_iv: "835280f3abcd0799fa0bf6da6eb0daa3"
server_key_version: "encryptionkey@0"
I20260809 22:53:09.873075 26101 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315935617919-15814-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315935617919-15814-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315935617919-15814-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:53:09.891571 26101 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:53:09.895816 26101 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:53:09.896106 26101 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:53:09.897045 26101 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:53:09.898412 26101 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:53:09.898499 26101 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:53:09.898629 26101 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:53:09.898691 26101 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:53:09.930805 26101 rpc_server.cc:307] RPC server started. Bound to: 127.15.113.130:37171
I20260809 22:53:09.930850 26299 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.113.130:37171 every 8 connection(s)
I20260809 22:53:09.932021 26101 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315935617919-15814-0/minicluster-data/ts-1/data/info.pb
I20260809 22:53:09.935310 15814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskItYPJR/build/asan/bin/kudu as pid 26101
I20260809 22:53:09.935494 15814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315935617919-15814-0/minicluster-data/ts-1/wal/instance
I20260809 22:53:09.936064 15814 external_mini_cluster.cc:1428] Setting key c7083af0569dfa7af6ee130d6ec58d18
Aug 09 22:53:09 dist-test-slave-ncp9 krb5kdc[25693](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315989, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.113.130@KRBTEST.COM for oryx/127.15.113.190@KRBTEST.COM
I20260809 22:53:09.953732 26095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:09.934442 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.113.130:37439 (local address 127.15.113.190:42225)
0809 22:53:09.934612 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0809 22:53:09.934616 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 22:53:09.935940 (+ 1324us) server_negotiation.cc:407] Connection header received
0809 22:53:09.938161 (+ 2221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:53:09.938167 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:53:09.938231 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:53:09.938313 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:53:09.940286 (+ 1973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:09.941109 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:09.943220 (+ 2111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:09.943926 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:09.944036 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:09.948080 (+ 4044us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:53:09.948085 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:53:09.948089 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:53:09.948134 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:53:09.950839 (+ 2705us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:53:09.951485 (+ 646us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:53:09.951490 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:53:09.951493 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:53:09.951641 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:53:09.951999 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:53:09.952003 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:53:09.952006 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:53:09.952370 (+ 364us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:53:09.952538 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0809 22:53:09.953161 (+ 623us) server_negotiation.cc:299] Negotiation successful
0809 22:53:09.953387 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260809 22:53:09.954501 26305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:09.934889 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:42225 (local address 127.15.113.130:37439)
0809 22:53:09.935761 (+ 872us) negotiation.cc:107] Waiting for socket to connect
0809 22:53:09.935821 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0809 22:53:09.937886 (+ 2065us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:53:09.938595 (+ 709us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:53:09.938608 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:53:09.939314 (+ 706us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:53:09.940097 (+ 783us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:53:09.940123 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:53:09.941265 (+ 1142us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:53:09.941269 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:53:09.943025 (+ 1756us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:53:09.943040 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:53:09.944152 (+ 1112us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:53:09.944159 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:53:09.944429 (+ 270us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:09.945157 (+ 728us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:53:09.945191 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:53:09.947836 (+ 2645us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:53:09.950963 (+ 3127us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:53:09.950969 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:53:09.950981 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:53:09.951374 (+ 393us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:53:09.951730 (+ 356us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:53:09.951733 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:53:09.951736 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:53:09.951903 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:53:09.952495 (+ 592us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:53:09.952500 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:53:09.952798 (+ 298us) client_negotiation.cc:769] Sending connection context
0809 22:53:09.953050 (+ 252us) client_negotiation.cc:240] Negotiation successful
0809 22:53:09.953409 (+ 359us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":543,"thread_start_us":304,"threads_started":1}
I20260809 22:53:09.955894 26300 heartbeater.cc:347] Connected to a master server at 127.15.113.190:42225
I20260809 22:53:09.956166 26300 heartbeater.cc:464] Registering TS with master...
I20260809 22:53:09.956862 26300 heartbeater.cc:511] Master 127.15.113.190:42225 requested a full tablet report, sending...
I20260809 22:53:09.958264 25770 ts_manager.cc:194] Registered new tserver with Master: 200f1dd145df4993b427e20748be2114 (127.15.113.130:37171)
I20260809 22:53:09.959115 25770 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.15.113.130@KRBTEST.COM'} at 127.15.113.130:37439
WARNING: no policy specified for oryx/127.15.113.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.15.113.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.15.113.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315935617919-15814-0/krb5kdc/oryx_127.15.113.131.keytab.
Entry for principal oryx/127.15.113.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315935617919-15814-0/krb5kdc/oryx_127.15.113.131.keytab.
I20260809 22:53:09.992287 15814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskItYPJR/build/asan/bin/kudu
/tmp/dist-test-taskItYPJR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315935617919-15814-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315935617919-15814-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315935617919-15814-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315935617919-15814-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.113.131:0
--local_ip_for_outbound_sockets=127.15.113.131
--webserver_interface=127.15.113.131
--webserver_port=0
--tserver_master_addrs=127.15.113.190:42225
--keytab_file=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315935617919-15814-0/krb5kdc/oryx_127.15.113.131.keytab
--principal=oryx/127.15.113.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.113.148:37185
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315935617919-15814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315935617919-15814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315935617919-15814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:53:10.309015 26313 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:53:10.309275 26313 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:53:10.309369 26313 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:53:10.316409 26313 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:53:10.316501 26313 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:53:10.316613 26313 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.113.131
I20260809 22:53:10.325832 26313 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.113.148:37185
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315935617919-15814-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315935617919-15814-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315935617919-15814-0/krb5kdc/oryx_127.15.113.131.keytab
--principal=oryx/127.15.113.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.113.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315935617919-15814-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.113.131
--webserver_port=0
--tserver_master_addrs=127.15.113.190:42225
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.113.131
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315935617919-15814-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:53:10.326903 26313 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:53:10.328125 26313 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:53:10.340164 26322 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:53:10.340242 26321 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:53:10.340224 26327 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:53:10.340971 26313 server_base.cc:1034] running on GCE node
I20260809 22:53:10.341518 26313 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:53:10.342382 26313 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:53:10.343578 26313 hybrid_clock.cc:648] HybridClock initialized: now 1786315990343521 us; error 45 us; skew 500 ppm
I20260809 22:53:10.343861 26313 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:53:10 dist-test-slave-ncp9 krb5kdc[25693](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315990, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.113.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:53:10.351532 26313 init.cc:377] Logged in from keytab as oryx/127.15.113.131@KRBTEST.COM (short username oryx)
I20260809 22:53:10.352903 26313 webserver.cc:460] Webserver started at http://127.15.113.131:34131/ using document root <none> and password file <none>
I20260809 22:53:10.353529 26313 fs_manager.cc:359] Metadata directory not provided
I20260809 22:53:10.353652 26313 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:53:10.353926 26313 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:53:10.356688 26313 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315935617919-15814-0/minicluster-data/ts-2/data/instance:
uuid: "a0eed0b7d94e4c1f905abd17c6d0fd80"
format_stamp: "Formatted at 2026-08-09 22:53:10 on dist-test-slave-ncp9"
server_key: "8b80b276f3fbc09d710c9fd7a50a134c"
server_key_iv: "25092b0b69bef32fe7df021917b0f828"
server_key_version: "encryptionkey@0"
I20260809 22:53:10.357473 26313 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315935617919-15814-0/minicluster-data/ts-2/wal/instance:
uuid: "a0eed0b7d94e4c1f905abd17c6d0fd80"
format_stamp: "Formatted at 2026-08-09 22:53:10 on dist-test-slave-ncp9"
server_key: "8b80b276f3fbc09d710c9fd7a50a134c"
server_key_iv: "25092b0b69bef32fe7df021917b0f828"
server_key_version: "encryptionkey@0"
I20260809 22:53:10.362210 26313 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.003s
I20260809 22:53:10.365664 26335 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:53:10.366997 26313 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 22:53:10.367136 26313 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315935617919-15814-0/minicluster-data/ts-2/data,/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315935617919-15814-0/minicluster-data/ts-2/wal
uuid: "a0eed0b7d94e4c1f905abd17c6d0fd80"
format_stamp: "Formatted at 2026-08-09 22:53:10 on dist-test-slave-ncp9"
server_key: "8b80b276f3fbc09d710c9fd7a50a134c"
server_key_iv: "25092b0b69bef32fe7df021917b0f828"
server_key_version: "encryptionkey@0"
I20260809 22:53:10.367302 26313 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315935617919-15814-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315935617919-15814-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315935617919-15814-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:53:10.380311 26313 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:53:10.383782 26313 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:53:10.384027 26313 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:53:10.384869 26313 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:53:10.386112 26313 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:53:10.386188 26313 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:53:10.386304 26313 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:53:10.386363 26313 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:53:10.415673 26313 rpc_server.cc:307] RPC server started. Bound to: 127.15.113.131:32941
I20260809 22:53:10.415706 26522 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.113.131:32941 every 8 connection(s)
I20260809 22:53:10.416882 26313 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315935617919-15814-0/minicluster-data/ts-2/data/info.pb
I20260809 22:53:10.423239 15814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskItYPJR/build/asan/bin/kudu as pid 26313
I20260809 22:53:10.423413 15814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315935617919-15814-0/minicluster-data/ts-2/wal/instance
I20260809 22:53:10.423854 15814 external_mini_cluster.cc:1428] Setting key a1aa985cd9d1eab75b26b5fd8f203966
Aug 09 22:53:10 dist-test-slave-ncp9 krb5kdc[25693](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315990, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.113.131@KRBTEST.COM for oryx/127.15.113.190@KRBTEST.COM
I20260809 22:53:10.436167 26095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:10.419239 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.113.131:48131 (local address 127.15.113.190:42225)
0809 22:53:10.419416 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0809 22:53:10.419420 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 22:53:10.420648 (+ 1228us) server_negotiation.cc:407] Connection header received
0809 22:53:10.423124 (+ 2476us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:53:10.423130 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:53:10.423222 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:53:10.423336 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:53:10.425368 (+ 2032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:10.426262 (+ 894us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:10.427866 (+ 1604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:10.428279 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:10.428373 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:10.431525 (+ 3152us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:53:10.431529 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:53:10.431532 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:53:10.431564 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:53:10.433856 (+ 2292us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:53:10.434467 (+ 611us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:53:10.434470 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:53:10.434473 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:53:10.434596 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:53:10.434917 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:53:10.434919 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:53:10.434921 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:53:10.435151 (+ 230us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:53:10.435258 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0809 22:53:10.435783 (+ 525us) server_negotiation.cc:299] Negotiation successful
0809 22:53:10.435935 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260809 22:53:10.437055 26526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:10.419696 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:42225 (local address 127.15.113.131:48131)
0809 22:53:10.420468 (+ 772us) negotiation.cc:107] Waiting for socket to connect
0809 22:53:10.420525 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0809 22:53:10.422805 (+ 2280us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:53:10.423700 (+ 895us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:53:10.423711 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:53:10.424366 (+ 655us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:53:10.425182 (+ 816us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:53:10.425208 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:53:10.426405 (+ 1197us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:53:10.426410 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:53:10.427743 (+ 1333us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:53:10.427755 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:53:10.428473 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:53:10.428481 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:53:10.428713 (+ 232us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:10.429342 (+ 629us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:53:10.429364 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:53:10.431342 (+ 1978us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:53:10.433984 (+ 2642us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:53:10.433990 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:53:10.434001 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:53:10.434383 (+ 382us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:53:10.434668 (+ 285us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:53:10.434671 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:53:10.434673 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:53:10.434841 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:53:10.435233 (+ 392us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:53:10.435238 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:53:10.435504 (+ 266us) client_negotiation.cc:769] Sending connection context
0809 22:53:10.435695 (+ 191us) client_negotiation.cc:240] Negotiation successful
0809 22:53:10.436029 (+ 334us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":499,"thread_start_us":283,"threads_started":1}
I20260809 22:53:10.438271 26523 heartbeater.cc:347] Connected to a master server at 127.15.113.190:42225
I20260809 22:53:10.438547 26523 heartbeater.cc:464] Registering TS with master...
I20260809 22:53:10.439224 26523 heartbeater.cc:511] Master 127.15.113.190:42225 requested a full tablet report, sending...
I20260809 22:53:10.440495 25770 ts_manager.cc:194] Registered new tserver with Master: a0eed0b7d94e4c1f905abd17c6d0fd80 (127.15.113.131:32941)
I20260809 22:53:10.441298 25770 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.15.113.131@KRBTEST.COM'} at 127.15.113.131:48131
I20260809 22:53:10.452739 15814 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 22:53:10.452860 26093 heartbeater.cc:503] Master 127.15.113.190:42225 was elected leader, sending a full tablet report...
WARNING: no policy specified for oryx/127.15.113.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.15.113.189@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.15.113.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315935617919-15814-0/krb5kdc/oryx_127.15.113.189.keytab.
Entry for principal oryx/127.15.113.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315935617919-15814-0/krb5kdc/oryx_127.15.113.189.keytab.
I20260809 22:53:10.512281 15814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskItYPJR/build/asan/bin/kudu
/tmp/dist-test-taskItYPJR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315935617919-15814-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315935617919-15814-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315935617919-15814-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315935617919-15814-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.113.189:41115
--webserver_interface=127.15.113.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315935617919-15814-0/krb5kdc/oryx_127.15.113.189.keytab
--principal=oryx/127.15.113.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.113.148:37185
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.113.190:42225,127.15.113.189:41115
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315935617919-15814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315935617919-15814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315935617919-15814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:53:10.834774 26529 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:53:10.835091 26529 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:53:10.835227 26529 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:53:10.842161 26529 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 22:53:10.842279 26529 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:53:10.842357 26529 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:53:10.842437 26529 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 22:53:10.842496 26529 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 22:53:10.851818 26529 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.113.148:37185
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315935617919-15814-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315935617919-15814-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.15.113.190:42225,127.15.113.189:41115
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315935617919-15814-0/krb5kdc/oryx_127.15.113.189.keytab
--principal=oryx/127.15.113.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.113.189:41115
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315935617919-15814-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.15.113.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315935617919-15814-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
W20260809 22:53:10.852157 26529 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.15.113.190:42225,127.15.113.189:41115'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260809 22:53:10.853150 26529 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:53:10.854386 26529 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:53:10.865298 26537 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:53:10.865345 26539 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:53:10.865443 26542 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:53:10.866811 26529 server_base.cc:1034] running on GCE node
I20260809 22:53:10.867504 26529 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:53:10.869072 26529 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:53:10.870299 26529 hybrid_clock.cc:648] HybridClock initialized: now 1786315990870227 us; error 49 us; skew 500 ppm
I20260809 22:53:10.870642 26529 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:53:10 dist-test-slave-ncp9 krb5kdc[25693](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315990, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.113.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:53:10.873848 26529 init.cc:377] Logged in from keytab as oryx/127.15.113.189@KRBTEST.COM (short username oryx)
I20260809 22:53:10.875129 26529 webserver.cc:460] Webserver started at http://127.15.113.189:37779/ using document root <none> and password file <none>
I20260809 22:53:10.875768 26529 fs_manager.cc:359] Metadata directory not provided
I20260809 22:53:10.875867 26529 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:53:10.876206 26529 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:53:10.879204 26529 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315935617919-15814-0/minicluster-data/master-1/data/instance:
uuid: "87d4ebb338564421816a43cffeb42715"
format_stamp: "Formatted at 2026-08-09 22:53:10 on dist-test-slave-ncp9"
server_key: "3da5eba2098515b68c7f257bc0afed40"
server_key_iv: "638646251a8062994ec0dc395ab0c4c8"
server_key_version: "encryptionkey@0"
I20260809 22:53:10.879925 26529 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315935617919-15814-0/minicluster-data/master-1/wal/instance:
uuid: "87d4ebb338564421816a43cffeb42715"
format_stamp: "Formatted at 2026-08-09 22:53:10 on dist-test-slave-ncp9"
server_key: "3da5eba2098515b68c7f257bc0afed40"
server_key_iv: "638646251a8062994ec0dc395ab0c4c8"
server_key_version: "encryptionkey@0"
I20260809 22:53:10.884780 26529 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.005s
I20260809 22:53:10.887988 26550 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:53:10.889242 26529 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 22:53:10.889384 26529 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315935617919-15814-0/minicluster-data/master-1/data,/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315935617919-15814-0/minicluster-data/master-1/wal
uuid: "87d4ebb338564421816a43cffeb42715"
format_stamp: "Formatted at 2026-08-09 22:53:10 on dist-test-slave-ncp9"
server_key: "3da5eba2098515b68c7f257bc0afed40"
server_key_iv: "638646251a8062994ec0dc395ab0c4c8"
server_key_version: "encryptionkey@0"
I20260809 22:53:10.889569 26529 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315935617919-15814-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315935617919-15814-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315935617919-15814-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:53:10.916338 26529 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:53:10.920130 26529 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:53:10.920379 26529 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:53:10.937863 26529 rpc_server.cc:307] RPC server started. Bound to: 127.15.113.189:41115
I20260809 22:53:10.937901 26641 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.113.189:41115 every 8 connection(s)
I20260809 22:53:10.939108 26529 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315935617919-15814-0/minicluster-data/master-1/data/info.pb
I20260809 22:53:10.942649 26642 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:53:10.944326 15814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskItYPJR/build/asan/bin/kudu as pid 26529
I20260809 22:53:10.951499 26642 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.113.190" port: 42225 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 22:53:10 dist-test-slave-ncp9 krb5kdc[25693](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315988, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.15.113.189@KRBTEST.COM
I20260809 22:53:10.961501 26300 heartbeater.cc:503] Master 127.15.113.190:42225 was elected leader, sending a full tablet report...
Aug 09 22:53:10 dist-test-slave-ncp9 krb5kdc[25693](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315990, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.113.189@KRBTEST.COM for oryx/127.15.113.190@KRBTEST.COM
I20260809 22:53:10.967306 26647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:10.946386 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50680 (local address 127.15.113.189:41115)
0809 22:53:10.947116 (+ 730us) server_negotiation.cc:206] Beginning negotiation
0809 22:53:10.947127 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 22:53:10.947172 (+ 45us) server_negotiation.cc:407] Connection header received
0809 22:53:10.948712 (+ 1540us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:53:10.948736 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:53:10.949160 (+ 424us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:53:10.949459 (+ 299us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:53:10.951227 (+ 1768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:10.952536 (+ 1309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:10.954143 (+ 1607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:10.955193 (+ 1050us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:10.955337 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:10.959845 (+ 4508us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:53:10.959855 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:53:10.959869 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:53:10.959921 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:53:10.963411 (+ 3490us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:53:10.964010 (+ 599us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:53:10.964018 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:53:10.964026 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:53:10.964183 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:53:10.964551 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:53:10.964556 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:53:10.964560 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:53:10.965108 (+ 548us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:53:10.965400 (+ 292us) server_negotiation.cc:1035] Waiting for connection context
0809 22:53:10.965870 (+ 470us) server_negotiation.cc:299] Negotiation successful
0809 22:53:10.966262 (+ 392us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":532,"thread_start_us":324,"threads_started":1}
I20260809 22:53:10.969271 26653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:10.954048 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:42225 (local address 127.0.0.1:52176)
0809 22:53:10.954897 (+ 849us) negotiation.cc:107] Waiting for socket to connect
0809 22:53:10.954933 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0809 22:53:10.955074 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:53:10.955471 (+ 397us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:53:10.955478 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:53:10.956065 (+ 587us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:53:10.956305 (+ 240us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:53:10.956314 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:53:10.957757 (+ 1443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:53:10.957760 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:53:10.958750 (+ 990us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:53:10.958763 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:53:10.959704 (+ 941us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:53:10.959711 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:53:10.959895 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:10.960501 (+ 606us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:53:10.960520 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:53:10.963349 (+ 2829us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:53:10.966813 (+ 3464us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:53:10.966825 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:53:10.966832 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:53:10.967263 (+ 431us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:53:10.967654 (+ 391us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:53:10.967659 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:53:10.967662 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:53:10.967810 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:53:10.968375 (+ 565us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:53:10.968383 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:53:10.968561 (+ 178us) client_negotiation.cc:769] Sending connection context
0809 22:53:10.968713 (+ 152us) client_negotiation.cc:240] Negotiation successful
0809 22:53:10.968977 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":654,"thread_start_us":519,"threads_started":1}
I20260809 22:53:10.969451 26649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:10.954080 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52176 (local address 127.15.113.190:42225)
0809 22:53:10.954544 (+ 464us) server_negotiation.cc:206] Beginning negotiation
0809 22:53:10.954550 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:53:10.955026 (+ 476us) server_negotiation.cc:407] Connection header received
0809 22:53:10.955200 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:53:10.955204 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:53:10.955288 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:53:10.955400 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:53:10.956463 (+ 1063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:10.957639 (+ 1176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:10.958894 (+ 1255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:10.959514 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:10.959657 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:10.963515 (+ 3858us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:53:10.963520 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:53:10.963523 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:53:10.963552 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:53:10.966665 (+ 3113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:53:10.967381 (+ 716us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:53:10.967386 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:53:10.967388 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:53:10.967536 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:53:10.967913 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:53:10.967919 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:53:10.967922 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:53:10.968237 (+ 315us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:53:10.968384 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0809 22:53:10.968850 (+ 466us) server_negotiation.cc:299] Negotiation successful
0809 22:53:10.969105 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":342,"thread_start_us":231,"threads_started":1}
I20260809 22:53:10.971021 26642 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.113.189" port: 41115 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 22:53:10 dist-test-slave-ncp9 krb5kdc[25693](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315990, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.113.189@KRBTEST.COM for oryx/127.15.113.189@KRBTEST.COM
I20260809 22:53:10.982115 26653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:10.971598 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.189:41115 (local address 127.0.0.1:50688)
0809 22:53:10.971768 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0809 22:53:10.971785 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 22:53:10.971870 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:53:10.972361 (+ 491us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:53:10.972366 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:53:10.972733 (+ 367us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:53:10.972856 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:53:10.972864 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:53:10.973835 (+ 971us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:53:10.973839 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:53:10.974675 (+ 836us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:53:10.974685 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:53:10.975293 (+ 608us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:53:10.975298 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:53:10.975466 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:10.975902 (+ 436us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:53:10.975917 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:53:10.977850 (+ 1933us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:53:10.980209 (+ 2359us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:53:10.980212 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:53:10.980215 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:53:10.980556 (+ 341us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:53:10.980832 (+ 276us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:53:10.980835 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:53:10.980837 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:53:10.981002 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:53:10.981470 (+ 468us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:53:10.981473 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:53:10.981585 (+ 112us) client_negotiation.cc:769] Sending connection context
0809 22:53:10.981692 (+ 107us) client_negotiation.cc:240] Negotiation successful
0809 22:53:10.981889 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":44}
I20260809 22:53:10.982278 26647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:10.972021 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50688 (local address 127.15.113.189:41115)
0809 22:53:10.972158 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0809 22:53:10.972161 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 22:53:10.972173 (+ 12us) server_negotiation.cc:407] Connection header received
0809 22:53:10.972205 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:53:10.972208 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:53:10.972252 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:53:10.972324 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:53:10.973012 (+ 688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:10.973706 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:10.974794 (+ 1088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:10.975170 (+ 376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:10.975271 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:10.978004 (+ 2733us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:53:10.978009 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:53:10.978012 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:53:10.978039 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:53:10.980100 (+ 2061us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:53:10.980654 (+ 554us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:53:10.980658 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:53:10.980660 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:53:10.980759 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:53:10.981108 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:53:10.981112 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:53:10.981114 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:53:10.981379 (+ 265us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:53:10.981479 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0809 22:53:10.981775 (+ 296us) server_negotiation.cc:299] Negotiation successful
0809 22:53:10.981994 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":59,"server-negotiator.queue_time_us":50}
I20260809 22:53:10.986673 26642 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 87d4ebb338564421816a43cffeb42715: Bootstrap starting.
I20260809 22:53:10.989746 26642 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 87d4ebb338564421816a43cffeb42715: Neither blocks nor log segments found. Creating new log.
I20260809 22:53:10.990757 26642 log.cc:824] T 00000000000000000000000000000000 P 87d4ebb338564421816a43cffeb42715: Log is configured to *not* fsync() on all Append() calls
I20260809 22:53:10.993428 26642 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 87d4ebb338564421816a43cffeb42715: No bootstrap required, opened a new log
I20260809 22:53:11.000203 26642 raft_consensus.cc:348] T 00000000000000000000000000000000 P 87d4ebb338564421816a43cffeb42715 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "460ec5412d65461a811d766224ae4ab6" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 42225 } } peers { permanent_uuid: "87d4ebb338564421816a43cffeb42715" member_type: VOTER last_known_addr { host: "127.15.113.189" port: 41115 } }
I20260809 22:53:11.000516 26642 raft_consensus.cc:374] T 00000000000000000000000000000000 P 87d4ebb338564421816a43cffeb42715 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:53:11.000608 26642 raft_consensus.cc:729] T 00000000000000000000000000000000 P 87d4ebb338564421816a43cffeb42715 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 87d4ebb338564421816a43cffeb42715, State: Initialized, Role: FOLLOWER
I20260809 22:53:11.001174 26642 consensus_queue.cc:260] T 00000000000000000000000000000000 P 87d4ebb338564421816a43cffeb42715 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "460ec5412d65461a811d766224ae4ab6" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 42225 } } peers { permanent_uuid: "87d4ebb338564421816a43cffeb42715" member_type: VOTER last_known_addr { host: "127.15.113.189" port: 41115 } }
I20260809 22:53:11.001971 26656 sys_catalog.cc:455] T 00000000000000000000000000000000 P 87d4ebb338564421816a43cffeb42715 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "460ec5412d65461a811d766224ae4ab6" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 42225 } } peers { permanent_uuid: "87d4ebb338564421816a43cffeb42715" member_type: VOTER last_known_addr { host: "127.15.113.189" port: 41115 } } }
I20260809 22:53:11.002278 26656 sys_catalog.cc:458] T 00000000000000000000000000000000 P 87d4ebb338564421816a43cffeb42715 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 22:53:11.002964 26642 sys_catalog.cc:564] T 00000000000000000000000000000000 P 87d4ebb338564421816a43cffeb42715 [sys.catalog]: configured and running, proceeding with master startup.
W20260809 22:53:11.010946 26673 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 87d4ebb338564421816a43cffeb42715: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 22:53:11.011080 26673 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 22:53:11.012864 26529 master_runner.cc:428] Detected that this master 87d4ebb338564421816a43cffeb42715 is joining an existing cluster
I20260809 22:53:11.012984 26529 master_runner.cc:432] Initiating AddMaster RPC to add 127.15.113.189:41115
I20260809 22:53:11.032389 26647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:11.016555 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50692 (local address 127.15.113.189:41115)
0809 22:53:11.016754 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0809 22:53:11.016760 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:53:11.017904 (+ 1144us) server_negotiation.cc:407] Connection header received
0809 22:53:11.018088 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:53:11.018093 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:53:11.018175 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:53:11.018235 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:53:11.019758 (+ 1523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:11.020777 (+ 1019us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:11.022728 (+ 1951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:11.023495 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:11.023606 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:11.026029 (+ 2423us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:53:11.026035 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:53:11.026038 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:53:11.026067 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:53:11.029341 (+ 3274us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:53:11.030170 (+ 829us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:53:11.030175 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:53:11.030178 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:53:11.030320 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:53:11.030715 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:53:11.030720 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:53:11.030722 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:53:11.031047 (+ 325us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:53:11.031202 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0809 22:53:11.031654 (+ 452us) server_negotiation.cc:299] Negotiation successful
0809 22:53:11.031880 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":75,"server-negotiator.queue_time_us":68}
I20260809 22:53:11.032943 26649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:11.016577 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52190 (local address 127.15.113.190:42225)
0809 22:53:11.016739 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0809 22:53:11.016745 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:53:11.017539 (+ 794us) server_negotiation.cc:407] Connection header received
0809 22:53:11.017833 (+ 294us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:53:11.017838 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:53:11.017928 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:53:11.018018 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:53:11.019168 (+ 1150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:11.019976 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:11.022039 (+ 2063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:11.022699 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:11.022801 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:11.025738 (+ 2937us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:53:11.025746 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:53:11.025751 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:53:11.025792 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:53:11.029285 (+ 3493us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:53:11.030184 (+ 899us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:53:11.030190 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:53:11.030193 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:53:11.030346 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:53:11.030820 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:53:11.030826 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:53:11.030829 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:53:11.031203 (+ 374us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:53:11.031335 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0809 22:53:11.032012 (+ 677us) server_negotiation.cc:299] Negotiation successful
0809 22:53:11.032272 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260809 22:53:11.033361 26681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:11.017133 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.189:41115 (local address 127.0.0.1:50692)
0809 22:53:11.017798 (+ 665us) negotiation.cc:107] Waiting for socket to connect
0809 22:53:11.017819 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0809 22:53:11.017948 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:53:11.018430 (+ 482us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:53:11.018434 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:53:11.019272 (+ 838us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:53:11.019607 (+ 335us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:53:11.019621 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:53:11.021004 (+ 1383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:53:11.021009 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:53:11.022563 (+ 1554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:53:11.022578 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:53:11.023855 (+ 1277us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:53:11.023861 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:53:11.024047 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:11.024627 (+ 580us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:53:11.024654 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:53:11.025869 (+ 1215us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:53:11.029476 (+ 3607us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:53:11.029480 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:53:11.029483 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:53:11.030055 (+ 572us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:53:11.030436 (+ 381us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:53:11.030440 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:53:11.030446 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:53:11.030610 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:53:11.031268 (+ 658us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:53:11.031272 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:53:11.031401 (+ 129us) client_negotiation.cc:769] Sending connection context
0809 22:53:11.032649 (+ 1248us) client_negotiation.cc:240] Negotiation successful
0809 22:53:11.032943 (+ 294us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":525,"mutex_wait_us":35,"thread_start_us":332,"threads_started":1}
I20260809 22:53:11.034400 26680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:11.016655 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:42225 (local address 127.0.0.1:52190)
0809 22:53:11.017432 (+ 777us) negotiation.cc:107] Waiting for socket to connect
0809 22:53:11.017455 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0809 22:53:11.017653 (+ 198us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:53:11.018050 (+ 397us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:53:11.018055 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:53:11.018608 (+ 553us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:53:11.019025 (+ 417us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:53:11.019035 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:53:11.020120 (+ 1085us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:53:11.020126 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:53:11.021861 (+ 1735us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:53:11.021877 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:53:11.023093 (+ 1216us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:53:11.023099 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:53:11.023431 (+ 332us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:11.024100 (+ 669us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:53:11.024133 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:53:11.025513 (+ 1380us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:53:11.029437 (+ 3924us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:53:11.029446 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:53:11.029450 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:53:11.030033 (+ 583us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:53:11.030504 (+ 471us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:53:11.030512 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:53:11.030515 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:53:11.030681 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:53:11.031416 (+ 735us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:53:11.031420 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:53:11.031706 (+ 286us) client_negotiation.cc:769] Sending connection context
0809 22:53:11.033590 (+ 1884us) client_negotiation.cc:240] Negotiation successful
0809 22:53:11.033918 (+ 328us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":618,"mutex_wait_us":74,"spinlock_wait_cycles":29952,"thread_start_us":408,"threads_started":1}
Aug 09 22:53:11 dist-test-slave-ncp9 krb5kdc[25693](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315988, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.113.190@KRBTEST.COM for oryx/127.15.113.189@KRBTEST.COM
I20260809 22:53:11.054739 26647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:11.040046 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50694 (local address 127.15.113.189:41115)
0809 22:53:11.040196 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0809 22:53:11.040199 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 22:53:11.041069 (+ 870us) server_negotiation.cc:407] Connection header received
0809 22:53:11.041343 (+ 274us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:53:11.041349 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:53:11.041425 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:53:11.041515 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:53:11.042970 (+ 1455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:11.043691 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:11.045284 (+ 1593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:11.045953 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:11.046062 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:11.049557 (+ 3495us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:53:11.049562 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:53:11.049565 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:53:11.049592 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:53:11.052168 (+ 2576us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:53:11.052838 (+ 670us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:53:11.052842 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:53:11.052845 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:53:11.053005 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:53:11.053371 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:53:11.053375 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:53:11.053377 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:53:11.053606 (+ 229us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:53:11.053733 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0809 22:53:11.054217 (+ 484us) server_negotiation.cc:299] Negotiation successful
0809 22:53:11.054402 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260809 22:53:11.054792 26683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:11.040065 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.189:41115 (local address 127.0.0.1:50694)
0809 22:53:11.040758 (+ 693us) negotiation.cc:107] Waiting for socket to connect
0809 22:53:11.040790 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0809 22:53:11.041166 (+ 376us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:53:11.041571 (+ 405us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:53:11.041578 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:53:11.042571 (+ 993us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:53:11.042835 (+ 264us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:53:11.042846 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:53:11.043825 (+ 979us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:53:11.043830 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:53:11.045134 (+ 1304us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:53:11.045148 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:53:11.046107 (+ 959us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:53:11.046114 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:53:11.046302 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:11.046886 (+ 584us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:53:11.046905 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:53:11.049379 (+ 2474us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:53:11.052287 (+ 2908us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:53:11.052300 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:53:11.052309 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:53:11.052714 (+ 405us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:53:11.053110 (+ 396us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:53:11.053115 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:53:11.053119 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:53:11.053258 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:53:11.053733 (+ 475us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:53:11.053741 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:53:11.053907 (+ 166us) client_negotiation.cc:769] Sending connection context
0809 22:53:11.054068 (+ 161us) client_negotiation.cc:240] Negotiation successful
0809 22:53:11.054376 (+ 308us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":497,"thread_start_us":355,"threads_started":1}
WARNING: no policy specified for oryx/127.15.113.188@KRBTEST.COM; defaulting to no policy
I20260809 22:53:11.056797 25770 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.15.113.189:41115: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "87d4ebb338564421816a43cffeb42715"
member_type: NON_VOTER
last_known_addr {
host: "127.15.113.189"
port: 41115
}
attrs {
promote: true
}
}
dest_uuid: "460ec5412d65461a811d766224ae4ab6"
cas_config_opid_index: -1
I20260809 22:53:11.058151 25770 consensus_queue.cc:237] T 00000000000000000000000000000000 P 460ec5412d65461a811d766224ae4ab6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "460ec5412d65461a811d766224ae4ab6" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 42225 } } peers { permanent_uuid: "87d4ebb338564421816a43cffeb42715" member_type: NON_VOTER last_known_addr { host: "127.15.113.189" port: 41115 } attrs { promote: true } }
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.15.113.188@KRBTEST.COM" created.
I20260809 22:53:11.063676 26685 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 460ec5412d65461a811d766224ae4ab6 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 87d4ebb338564421816a43cffeb42715 (127.15.113.189) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "460ec5412d65461a811d766224ae4ab6" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 42225 } } peers { permanent_uuid: "87d4ebb338564421816a43cffeb42715" member_type: NON_VOTER last_known_addr { host: "127.15.113.189" port: 41115 } attrs { promote: true } } }
I20260809 22:53:11.072129 26593 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 87d4ebb338564421816a43cffeb42715 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:53:11.073146 26685 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.15.113.189:41115
I20260809 22:53:11.074070 26529 master.cc:537] Master@127.15.113.189:41115 shutting down...
I20260809 22:53:11.074051 26686 sys_catalog.cc:455] T 00000000000000000000000000000000 P 460ec5412d65461a811d766224ae4ab6 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "460ec5412d65461a811d766224ae4ab6" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "460ec5412d65461a811d766224ae4ab6" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 42225 } } peers { permanent_uuid: "87d4ebb338564421816a43cffeb42715" member_type: NON_VOTER last_known_addr { host: "127.15.113.189" port: 41115 } attrs { promote: true } } }
I20260809 22:53:11.075048 26686 sys_catalog.cc:458] T 00000000000000000000000000000000 P 460ec5412d65461a811d766224ae4ab6 [sys.catalog]: This master's current role is: LEADER
I20260809 22:53:11.076018 26593 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 87d4ebb338564421816a43cffeb42715 [term 1 FOLLOWER]: Refusing update from remote peer 460ec5412d65461a811d766224ae4ab6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260809 22:53:11.077364 26685 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 460ec5412d65461a811d766224ae4ab6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "87d4ebb338564421816a43cffeb42715" member_type: NON_VOTER last_known_addr { host: "127.15.113.189" port: 41115 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
W20260809 22:53:11.080816 25751 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
I20260809 22:53:11.092113 26529 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 87d4ebb338564421816a43cffeb42715 [term 1 FOLLOWER]: Raft consensus shutting down.
W20260809 22:53:11.092154 25751 consensus_peers.cc:487] T 00000000000000000000000000000000 P 460ec5412d65461a811d766224ae4ab6 -> Peer 87d4ebb338564421816a43cffeb42715 (127.15.113.189:41115): Couldn't send request to peer 87d4ebb338564421816a43cffeb42715. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260809 22:53:11.092696 26529 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 87d4ebb338564421816a43cffeb42715 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 22:53:11.093041 26529 tablet_replica.cc:330] T 00000000000000000000000000000000 P 87d4ebb338564421816a43cffeb42715: stopping tablet replica
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.15.113.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315935617919-15814-0/krb5kdc/oryx_127.15.113.188.keytab.
Entry for principal oryx/127.15.113.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315935617919-15814-0/krb5kdc/oryx_127.15.113.188.keytab.
I20260809 22:53:11.104219 15814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskItYPJR/build/asan/bin/kudu
/tmp/dist-test-taskItYPJR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315935617919-15814-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315935617919-15814-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315935617919-15814-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315935617919-15814-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.113.188:43669
--webserver_interface=127.15.113.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315935617919-15814-0/krb5kdc/oryx_127.15.113.188.keytab
--principal=oryx/127.15.113.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.113.148:37185
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.113.190:42225,127.15.113.189:41115,127.15.113.188:43669
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315935617919-15814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315935617919-15814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315935617919-15814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 22:53:11.110258 26529 master.cc:552] Master@127.15.113.189:41115 shutdown complete.
I20260809 22:53:11.110399 26529 master_runner.cc:305] Clearing existing system tablet
I20260809 22:53:11.110935 26529 fs_manager.cc:359] Metadata directory not provided
I20260809 22:53:11.111033 26529 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:53:11.114004 26529 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.000s sys 0.004s
I20260809 22:53:11.115612 26695 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:53:11.115935 26529 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315935617919-15814-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:53:11.116199 26529 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 22:53:11.116287 26529 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315935617919-15814-0/minicluster-data/master-1/data,/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315935617919-15814-0/minicluster-data/master-1/wal
uuid: "87d4ebb338564421816a43cffeb42715"
format_stamp: "Formatted at 2026-08-09 22:53:10 on dist-test-slave-ncp9"
server_key: "3da5eba2098515b68c7f257bc0afed40"
server_key_iv: "638646251a8062994ec0dc395ab0c4c8"
server_key_version: "encryptionkey@0"
I20260809 22:53:11.119683 26529 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 87d4ebb338564421816a43cffeb42715: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 22:53:11.124202 26529 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 87d4ebb338564421816a43cffeb42715: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260809 22:53:11.124363 26529 log.cc:1196] T 00000000000000000000000000000000 P 87d4ebb338564421816a43cffeb42715: Deleting WAL directory at /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315935617919-15814-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260809 22:53:11.124742 26529 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 87d4ebb338564421816a43cffeb42715: Deleting consensus metadata
I20260809 22:53:11.125006 26529 master_runner.cc:315] Copying system tablet from 127.15.113.190:42225
I20260809 22:53:11.127758 26529 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 87d4ebb338564421816a43cffeb42715: tablet copy: Beginning tablet copy session from remote peer at address 127.15.113.190:42225
I20260809 22:53:11.144280 26649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:11.128751 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52192 (local address 127.15.113.190:42225)
0809 22:53:11.128979 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0809 22:53:11.128985 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:53:11.133450 (+ 4465us) server_negotiation.cc:407] Connection header received
0809 22:53:11.133651 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:53:11.133655 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:53:11.133724 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:53:11.133809 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:53:11.134673 (+ 864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:11.135415 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:11.136659 (+ 1244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:11.137154 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:11.137271 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:11.139219 (+ 1948us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:53:11.139224 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:53:11.139227 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:53:11.139260 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:53:11.141721 (+ 2461us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:53:11.142368 (+ 647us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:53:11.142374 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:53:11.142377 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:53:11.142525 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:53:11.142893 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:53:11.142897 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:53:11.142899 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:53:11.143161 (+ 262us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:53:11.143293 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0809 22:53:11.143715 (+ 422us) server_negotiation.cc:299] Negotiation successful
0809 22:53:11.143921 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260809 22:53:11.144315 26705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:11.128635 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:42225 (local address 127.0.0.1:52192)
0809 22:53:11.133326 (+ 4691us) negotiation.cc:107] Waiting for socket to connect
0809 22:53:11.133343 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 22:53:11.133497 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:53:11.133856 (+ 359us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:53:11.133861 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:53:11.134300 (+ 439us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:53:11.134509 (+ 209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:53:11.134518 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:53:11.135559 (+ 1041us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:53:11.135563 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:53:11.136506 (+ 943us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:53:11.136519 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:53:11.137307 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:53:11.137313 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:53:11.137505 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:11.138298 (+ 793us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:53:11.138316 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:53:11.139025 (+ 709us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:53:11.141843 (+ 2818us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:53:11.141852 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:53:11.141856 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:53:11.142224 (+ 368us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:53:11.142652 (+ 428us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:53:11.142655 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:53:11.142657 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:53:11.142786 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:53:11.143277 (+ 491us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:53:11.143282 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:53:11.143440 (+ 158us) client_negotiation.cc:769] Sending connection context
0809 22:53:11.143575 (+ 135us) client_negotiation.cc:240] Negotiation successful
0809 22:53:11.143891 (+ 316us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4573,"thread_start_us":4421,"threads_started":1}
I20260809 22:53:11.145483 25806 tablet_copy_service.cc:140] P 460ec5412d65461a811d766224ae4ab6: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 87d4ebb338564421816a43cffeb42715 ({username='oryx', principal='oryx/127.15.113.189@KRBTEST.COM'} at 127.0.0.1:52192)
I20260809 22:53:11.145882 25806 tablet_copy_service.cc:161] P 460ec5412d65461a811d766224ae4ab6: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 87d4ebb338564421816a43cffeb42715 at {username='oryx', principal='oryx/127.15.113.189@KRBTEST.COM'} at 127.0.0.1:52192: session id = 87d4ebb338564421816a43cffeb42715-00000000000000000000000000000000
I20260809 22:53:11.149255 25806 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 460ec5412d65461a811d766224ae4ab6: Tablet Copy: opened 0 blocks and 1 log segments
I20260809 22:53:11.151582 26529 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260809 22:53:11.156250 26529 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 87d4ebb338564421816a43cffeb42715: tablet copy: Starting download of 0 data blocks...
I20260809 22:53:11.156574 26529 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 87d4ebb338564421816a43cffeb42715: tablet copy: Starting download of 1 WAL segments...
I20260809 22:53:11.159513 26529 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 87d4ebb338564421816a43cffeb42715: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260809 22:53:11.162751 25806 tablet_copy_service.cc:342] P 460ec5412d65461a811d766224ae4ab6: Request end of tablet copy session 87d4ebb338564421816a43cffeb42715-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.15.113.189@KRBTEST.COM'} at 127.0.0.1:52192
I20260809 22:53:11.162952 25806 tablet_copy_service.cc:433] P 460ec5412d65461a811d766224ae4ab6: ending tablet copy session 87d4ebb338564421816a43cffeb42715-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 87d4ebb338564421816a43cffeb42715
I20260809 22:53:11.174685 26529 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:53:11.178771 26706 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:53:11.185227 26707 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:53:11.196971 26709 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:53:11.198159 26529 server_base.cc:1034] running on GCE node
I20260809 22:53:11.198603 26529 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:53:11.199360 26529 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:53:11.200651 26529 hybrid_clock.cc:648] HybridClock initialized: now 1786315991200585 us; error 34 us; skew 500 ppm
I20260809 22:53:11.200941 26529 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:53:11 dist-test-slave-ncp9 krb5kdc[25693](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315991, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.113.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:53:11.203212 26529 init.cc:377] Logged in from keytab as oryx/127.15.113.189@KRBTEST.COM (short username oryx)
I20260809 22:53:11.205813 26529 webserver.cc:460] Webserver started at http://127.15.113.189:45827/ using document root <none> and password file <none>
I20260809 22:53:11.206218 26529 fs_manager.cc:359] Metadata directory not provided
I20260809 22:53:11.206301 26529 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:53:11.209535 26529 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260809 22:53:11.211613 26719 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:53:11.212054 26529 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.001s
I20260809 22:53:11.212217 26529 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315935617919-15814-0/minicluster-data/master-1/data,/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315935617919-15814-0/minicluster-data/master-1/wal
uuid: "87d4ebb338564421816a43cffeb42715"
format_stamp: "Formatted at 2026-08-09 22:53:10 on dist-test-slave-ncp9"
server_key: "3da5eba2098515b68c7f257bc0afed40"
server_key_iv: "638646251a8062994ec0dc395ab0c4c8"
server_key_version: "encryptionkey@0"
I20260809 22:53:11.212392 26529 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315935617919-15814-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315935617919-15814-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315935617919-15814-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:53:11.241277 26529 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:53:11.245355 26529 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:53:11.272308 26529 rpc_server.cc:307] RPC server started. Bound to: 127.15.113.189:41115
I20260809 22:53:11.272341 26801 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.113.189:41115 every 8 connection(s)
I20260809 22:53:11.273602 26529 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315935617919-15814-0/minicluster-data/master-1/data/info.pb
I20260809 22:53:11.278985 26803 sys_catalog.cc:263] Verifying existing consensus state
I20260809 22:53:11.282961 26803 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 87d4ebb338564421816a43cffeb42715: Bootstrap starting.
I20260809 22:53:11.300789 26803 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 87d4ebb338564421816a43cffeb42715: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 22:53:11.301383 26803 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 87d4ebb338564421816a43cffeb42715: Bootstrap complete.
I20260809 22:53:11.302564 26803 raft_consensus.cc:348] T 00000000000000000000000000000000 P 87d4ebb338564421816a43cffeb42715 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "460ec5412d65461a811d766224ae4ab6" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 42225 } } peers { permanent_uuid: "87d4ebb338564421816a43cffeb42715" member_type: NON_VOTER last_known_addr { host: "127.15.113.189" port: 41115 } attrs { promote: true } }
I20260809 22:53:11.302875 26803 raft_consensus.cc:729] T 00000000000000000000000000000000 P 87d4ebb338564421816a43cffeb42715 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 87d4ebb338564421816a43cffeb42715, State: Initialized, Role: LEARNER
I20260809 22:53:11.303227 26803 consensus_queue.cc:260] T 00000000000000000000000000000000 P 87d4ebb338564421816a43cffeb42715 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "460ec5412d65461a811d766224ae4ab6" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 42225 } } peers { permanent_uuid: "87d4ebb338564421816a43cffeb42715" member_type: NON_VOTER last_known_addr { host: "127.15.113.189" port: 41115 } attrs { promote: true } }
I20260809 22:53:11.304244 26807 sys_catalog.cc:455] T 00000000000000000000000000000000 P 87d4ebb338564421816a43cffeb42715 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "460ec5412d65461a811d766224ae4ab6" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 42225 } } peers { permanent_uuid: "87d4ebb338564421816a43cffeb42715" member_type: NON_VOTER last_known_addr { host: "127.15.113.189" port: 41115 } attrs { promote: true } } }
I20260809 22:53:11.304514 26803 sys_catalog.cc:564] T 00000000000000000000000000000000 P 87d4ebb338564421816a43cffeb42715 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 22:53:11.304715 26807 sys_catalog.cc:458] T 00000000000000000000000000000000 P 87d4ebb338564421816a43cffeb42715 [sys.catalog]: This master's current role is: LEARNER
I20260809 22:53:11.316735 26821 catalog_manager.cc:1259] Loaded cluster ID: 2dcfd0b4c82745d78a39e01701586553
I20260809 22:53:11.316845 26821 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 87d4ebb338564421816a43cffeb42715: loading cluster ID for follower catalog manager: success
I20260809 22:53:11.320519 26821 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 87d4ebb338564421816a43cffeb42715: acquiring CA information for follower catalog manager: success
I20260809 22:53:11.323114 26821 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 87d4ebb338564421816a43cffeb42715: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 22:53:11.445240 26523 heartbeater.cc:503] Master 127.15.113.190:42225 was elected leader, sending a full tablet report...
I20260809 22:53:11.512426 26683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:11.502171 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.189:41115 (local address 127.0.0.1:50700)
0809 22:53:11.502462 (+ 291us) negotiation.cc:107] Waiting for socket to connect
0809 22:53:11.502484 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0809 22:53:11.502624 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:53:11.504038 (+ 1414us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:53:11.504044 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:53:11.504074 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:53:11.504569 (+ 495us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:53:11.504584 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:53:11.506828 (+ 2244us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:53:11.506833 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:53:11.508798 (+ 1965us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:53:11.508816 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:53:11.511254 (+ 2438us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:53:11.511261 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:53:11.511775 (+ 514us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:11.511844 (+ 69us) client_negotiation.cc:769] Sending connection context
0809 22:53:11.511919 (+ 75us) client_negotiation.cc:240] Negotiation successful
0809 22:53:11.512059 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":152}
I20260809 22:53:11.514155 26822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:11.502343 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50700 (local address 127.15.113.189:41115)
0809 22:53:11.503694 (+ 1351us) server_negotiation.cc:206] Beginning negotiation
0809 22:53:11.503700 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:53:11.503723 (+ 23us) server_negotiation.cc:407] Connection header received
0809 22:53:11.503787 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:53:11.503791 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:53:11.503856 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:53:11.503963 (+ 107us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:53:11.504724 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:11.506618 (+ 1894us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:11.509055 (+ 2437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:11.510980 (+ 1925us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:11.512999 (+ 2019us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:11.513194 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0809 22:53:11.513247 (+ 53us) server_negotiation.cc:299] Negotiation successful
0809 22:53:11.513413 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1176,"thread_start_us":968,"threads_started":1}
I20260809 22:53:11.516392 26763 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 87d4ebb338564421816a43cffeb42715 [term 1 LEARNER]: Deduplicated request from leader. Original: 0.0->[1.1-1.5] Dedup: 1.5->[]
I20260809 22:53:11.517509 26807 sys_catalog.cc:455] T 00000000000000000000000000000000 P 87d4ebb338564421816a43cffeb42715 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 460ec5412d65461a811d766224ae4ab6. Latest consensus state: current_term: 1 leader_uuid: "460ec5412d65461a811d766224ae4ab6" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "460ec5412d65461a811d766224ae4ab6" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 42225 } } peers { permanent_uuid: "87d4ebb338564421816a43cffeb42715" member_type: NON_VOTER last_known_addr { host: "127.15.113.189" port: 41115 } attrs { promote: true } } }
I20260809 22:53:11.517935 26807 sys_catalog.cc:458] T 00000000000000000000000000000000 P 87d4ebb338564421816a43cffeb42715 [sys.catalog]: This master's current role is: LEARNER
W20260809 22:53:11.666733 26693 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:53:11.667136 26693 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:53:11.667374 26693 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:53:11.676693 26693 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 22:53:11.676823 26693 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:53:11.676936 26693 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:53:11.677043 26693 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 22:53:11.677124 26693 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 22:53:11.689141 26693 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.113.148:37185
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315935617919-15814-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315935617919-15814-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.15.113.190:42225,127.15.113.189:41115,127.15.113.188:43669
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315935617919-15814-0/krb5kdc/oryx_127.15.113.188.keytab
--principal=oryx/127.15.113.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.113.188:43669
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315935617919-15814-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.15.113.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315935617919-15814-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:53:11.690668 26693 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:53:11.692207 26693 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:53:11.706296 26829 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:53:11.707144 26830 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:53:11.707190 26835 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:53:11.708359 26693 server_base.cc:1034] running on GCE node
I20260809 22:53:11.709416 26693 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:53:11.710824 26693 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:53:11.712088 26693 hybrid_clock.cc:648] HybridClock initialized: now 1786315991711961 us; error 47 us; skew 500 ppm
I20260809 22:53:11.712608 26693 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:53:11 dist-test-slave-ncp9 krb5kdc[25693](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315991, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.113.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:53:11.716202 26693 init.cc:377] Logged in from keytab as oryx/127.15.113.188@KRBTEST.COM (short username oryx)
I20260809 22:53:11.718142 26693 webserver.cc:460] Webserver started at http://127.15.113.188:40577/ using document root <none> and password file <none>
I20260809 22:53:11.719002 26693 fs_manager.cc:359] Metadata directory not provided
I20260809 22:53:11.719225 26693 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:53:11.719605 26693 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:53:11.723359 26693 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315935617919-15814-0/minicluster-data/master-2/data/instance:
uuid: "70317072228d4a47877776b7fb4e21d1"
format_stamp: "Formatted at 2026-08-09 22:53:11 on dist-test-slave-ncp9"
server_key: "cf4e1571e35110a74dfd8be895685a4e"
server_key_iv: "173a607b5f31669f6c6495b1906827a1"
server_key_version: "encryptionkey@0"
I20260809 22:53:11.724166 26693 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315935617919-15814-0/minicluster-data/master-2/wal/instance:
uuid: "70317072228d4a47877776b7fb4e21d1"
format_stamp: "Formatted at 2026-08-09 22:53:11 on dist-test-slave-ncp9"
server_key: "cf4e1571e35110a74dfd8be895685a4e"
server_key_iv: "173a607b5f31669f6c6495b1906827a1"
server_key_version: "encryptionkey@0"
I20260809 22:53:11.730527 26693 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.008s sys 0.000s
I20260809 22:53:11.734964 26846 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:53:11.736337 26693 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 22:53:11.736554 26693 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315935617919-15814-0/minicluster-data/master-2/data,/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315935617919-15814-0/minicluster-data/master-2/wal
uuid: "70317072228d4a47877776b7fb4e21d1"
format_stamp: "Formatted at 2026-08-09 22:53:11 on dist-test-slave-ncp9"
server_key: "cf4e1571e35110a74dfd8be895685a4e"
server_key_iv: "173a607b5f31669f6c6495b1906827a1"
server_key_version: "encryptionkey@0"
I20260809 22:53:11.736845 26693 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315935617919-15814-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315935617919-15814-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315935617919-15814-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:53:11.755577 26693 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:53:11.760968 26693 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:53:11.761282 26693 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:53:11.803774 26693 rpc_server.cc:307] RPC server started. Bound to: 127.15.113.188:43669
I20260809 22:53:11.803946 26936 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.113.188:43669 every 8 connection(s)
I20260809 22:53:11.805714 26693 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315935617919-15814-0/minicluster-data/master-2/data/info.pb
I20260809 22:53:11.806282 15814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskItYPJR/build/asan/bin/kudu as pid 26693
I20260809 22:53:11.810979 26938 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:53:11.819135 26938 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.113.190" port: 42225 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 22:53:11 dist-test-slave-ncp9 krb5kdc[25693](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315988, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.15.113.188@KRBTEST.COM
Aug 09 22:53:11 dist-test-slave-ncp9 krb5kdc[25693](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315991, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.113.188@KRBTEST.COM for oryx/127.15.113.190@KRBTEST.COM
I20260809 22:53:11.839332 26941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:11.809418 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37790 (local address 127.15.113.188:43669)
0809 22:53:11.810652 (+ 1234us) server_negotiation.cc:206] Beginning negotiation
0809 22:53:11.810664 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 22:53:11.810709 (+ 45us) server_negotiation.cc:407] Connection header received
0809 22:53:11.813388 (+ 2679us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:53:11.813416 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:53:11.813880 (+ 464us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:53:11.814304 (+ 424us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:53:11.816046 (+ 1742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:11.817790 (+ 1744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:11.819351 (+ 1561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:11.820310 (+ 959us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:11.820483 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:11.827651 (+ 7168us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:53:11.827662 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:53:11.827676 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:53:11.827762 (+ 86us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:53:11.833583 (+ 5821us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:53:11.834629 (+ 1046us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:53:11.834638 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:53:11.834646 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:53:11.835017 (+ 371us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:53:11.835536 (+ 519us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:53:11.835542 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:53:11.835545 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:53:11.836290 (+ 745us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:53:11.836728 (+ 438us) server_negotiation.cc:1035] Waiting for connection context
0809 22:53:11.837217 (+ 489us) server_negotiation.cc:299] Negotiation successful
0809 22:53:11.837623 (+ 406us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":944,"thread_start_us":597,"threads_started":1}
I20260809 22:53:11.845049 26942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:11.821689 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:42225 (local address 127.0.0.1:52204)
0809 22:53:11.822798 (+ 1109us) negotiation.cc:107] Waiting for socket to connect
0809 22:53:11.822847 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0809 22:53:11.822989 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:53:11.823743 (+ 754us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:53:11.823750 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:53:11.824838 (+ 1088us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:53:11.825711 (+ 873us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:53:11.825725 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:53:11.827482 (+ 1757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:53:11.827486 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:53:11.829530 (+ 2044us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:53:11.829543 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:53:11.830622 (+ 1079us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:53:11.830631 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:53:11.830925 (+ 294us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:11.831755 (+ 830us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:53:11.831805 (+ 50us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:53:11.834990 (+ 3185us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:53:11.841782 (+ 6792us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:53:11.841792 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:53:11.841797 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:53:11.842455 (+ 658us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:53:11.842930 (+ 475us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:53:11.842935 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:53:11.842939 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:53:11.843217 (+ 278us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:53:11.843903 (+ 686us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:53:11.843909 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:53:11.844111 (+ 202us) client_negotiation.cc:769] Sending connection context
0809 22:53:11.844288 (+ 177us) client_negotiation.cc:240] Negotiation successful
0809 22:53:11.844511 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":827,"thread_start_us":556,"threads_started":1}
I20260809 22:53:11.845199 26943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:11.821909 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52204 (local address 127.15.113.190:42225)
0809 22:53:11.823396 (+ 1487us) server_negotiation.cc:206] Beginning negotiation
0809 22:53:11.823400 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 22:53:11.823417 (+ 17us) server_negotiation.cc:407] Connection header received
0809 22:53:11.823468 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:53:11.823472 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:53:11.823549 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:53:11.823636 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:53:11.825883 (+ 2247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:11.827255 (+ 1372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:11.829673 (+ 2418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:11.830332 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:11.830455 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:11.835208 (+ 4753us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:53:11.835213 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:53:11.835216 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:53:11.835248 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:53:11.841555 (+ 6307us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:53:11.842578 (+ 1023us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:53:11.842582 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:53:11.842585 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:53:11.842798 (+ 213us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:53:11.843358 (+ 560us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:53:11.843365 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:53:11.843370 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:53:11.843781 (+ 411us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:53:11.843936 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0809 22:53:11.844412 (+ 476us) server_negotiation.cc:299] Negotiation successful
0809 22:53:11.844609 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1345,"thread_start_us":1240,"threads_started":1}
I20260809 22:53:11.846539 26938 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.113.189" port: 41115 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 22:53:11 dist-test-slave-ncp9 krb5kdc[25693](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315991, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.113.188@KRBTEST.COM for oryx/127.15.113.189@KRBTEST.COM
I20260809 22:53:11.860726 26942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:11.847235 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.189:41115 (local address 127.0.0.1:50710)
0809 22:53:11.847518 (+ 283us) negotiation.cc:107] Waiting for socket to connect
0809 22:53:11.847541 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0809 22:53:11.847658 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:53:11.848082 (+ 424us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:53:11.848088 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:53:11.848764 (+ 676us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:53:11.849144 (+ 380us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:53:11.849161 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:53:11.850159 (+ 998us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:53:11.850164 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:53:11.851184 (+ 1020us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:53:11.851208 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:53:11.851886 (+ 678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:53:11.851891 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:53:11.852049 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:11.852653 (+ 604us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:53:11.852671 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:53:11.855331 (+ 2660us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:53:11.858693 (+ 3362us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:53:11.858697 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:53:11.858699 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:53:11.859092 (+ 393us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:53:11.859396 (+ 304us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:53:11.859399 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:53:11.859401 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:53:11.859562 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:53:11.860085 (+ 523us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:53:11.860089 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:53:11.860208 (+ 119us) client_negotiation.cc:769] Sending connection context
0809 22:53:11.860319 (+ 111us) client_negotiation.cc:240] Negotiation successful
0809 22:53:11.860516 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":59}
I20260809 22:53:11.860808 26822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:11.847645 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50710 (local address 127.15.113.189:41115)
0809 22:53:11.847816 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0809 22:53:11.847820 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 22:53:11.847835 (+ 15us) server_negotiation.cc:407] Connection header received
0809 22:53:11.847882 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:53:11.847885 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:53:11.847941 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:53:11.848022 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:53:11.849295 (+ 1273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:11.849991 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:11.851351 (+ 1360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:11.851752 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:11.851843 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:11.855516 (+ 3673us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:53:11.855521 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:53:11.855525 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:53:11.855558 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:53:11.858567 (+ 3009us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:53:11.859189 (+ 622us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:53:11.859193 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:53:11.859195 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:53:11.859313 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:53:11.859689 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:53:11.859693 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:53:11.859696 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:53:11.859959 (+ 263us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:53:11.860074 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0809 22:53:11.860386 (+ 312us) server_negotiation.cc:299] Negotiation successful
0809 22:53:11.860557 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260809 22:53:11.861945 26938 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.113.188" port: 43669 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 22:53:11 dist-test-slave-ncp9 krb5kdc[25693](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315991, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.113.188@KRBTEST.COM for oryx/127.15.113.188@KRBTEST.COM
I20260809 22:53:11.878835 26942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:11.862547 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.188:43669 (local address 127.0.0.1:37800)
0809 22:53:11.862700 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0809 22:53:11.862718 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 22:53:11.862788 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:53:11.863313 (+ 525us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:53:11.863318 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:53:11.863850 (+ 532us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:53:11.864100 (+ 250us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:53:11.864110 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:53:11.865315 (+ 1205us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:53:11.865319 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:53:11.866651 (+ 1332us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:53:11.866664 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:53:11.867526 (+ 862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:53:11.867534 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:53:11.867955 (+ 421us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:11.868551 (+ 596us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:53:11.868567 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:53:11.871377 (+ 2810us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:53:11.875545 (+ 4168us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:53:11.875551 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:53:11.875555 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:53:11.876149 (+ 594us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:53:11.876612 (+ 463us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:53:11.876617 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:53:11.876620 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:53:11.876964 (+ 344us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:53:11.877557 (+ 593us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:53:11.877561 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:53:11.877786 (+ 225us) client_negotiation.cc:769] Sending connection context
0809 22:53:11.877902 (+ 116us) client_negotiation.cc:240] Negotiation successful
0809 22:53:11.878153 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42}
I20260809 22:53:11.879115 26941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:11.862910 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37800 (local address 127.15.113.188:43669)
0809 22:53:11.863055 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0809 22:53:11.863059 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 22:53:11.863071 (+ 12us) server_negotiation.cc:407] Connection header received
0809 22:53:11.863102 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:53:11.863105 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:53:11.863158 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:53:11.863262 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:53:11.864228 (+ 966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:11.865192 (+ 964us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:11.866775 (+ 1583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:11.867318 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:11.867425 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:11.871576 (+ 4151us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:53:11.871581 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:53:11.871584 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:53:11.871611 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:53:11.875390 (+ 3779us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:53:11.876252 (+ 862us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:53:11.876257 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:53:11.876260 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:53:11.876484 (+ 224us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:53:11.877082 (+ 598us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:53:11.877086 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:53:11.877088 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:53:11.877446 (+ 358us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:53:11.877572 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0809 22:53:11.878043 (+ 471us) server_negotiation.cc:299] Negotiation successful
0809 22:53:11.878391 (+ 348us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":42,"server-negotiator.queue_time_us":61}
I20260809 22:53:11.885210 26938 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 70317072228d4a47877776b7fb4e21d1: Bootstrap starting.
I20260809 22:53:11.890085 26938 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 70317072228d4a47877776b7fb4e21d1: Neither blocks nor log segments found. Creating new log.
I20260809 22:53:11.892025 26938 log.cc:824] T 00000000000000000000000000000000 P 70317072228d4a47877776b7fb4e21d1: Log is configured to *not* fsync() on all Append() calls
I20260809 22:53:11.895884 26938 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 70317072228d4a47877776b7fb4e21d1: No bootstrap required, opened a new log
I20260809 22:53:11.906999 26938 raft_consensus.cc:348] T 00000000000000000000000000000000 P 70317072228d4a47877776b7fb4e21d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "460ec5412d65461a811d766224ae4ab6" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 42225 } } peers { permanent_uuid: "87d4ebb338564421816a43cffeb42715" member_type: VOTER last_known_addr { host: "127.15.113.189" port: 41115 } } peers { permanent_uuid: "70317072228d4a47877776b7fb4e21d1" member_type: VOTER last_known_addr { host: "127.15.113.188" port: 43669 } }
I20260809 22:53:11.907487 26938 raft_consensus.cc:374] T 00000000000000000000000000000000 P 70317072228d4a47877776b7fb4e21d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:53:11.907684 26938 raft_consensus.cc:729] T 00000000000000000000000000000000 P 70317072228d4a47877776b7fb4e21d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 70317072228d4a47877776b7fb4e21d1, State: Initialized, Role: FOLLOWER
I20260809 22:53:11.908390 26938 consensus_queue.cc:260] T 00000000000000000000000000000000 P 70317072228d4a47877776b7fb4e21d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "460ec5412d65461a811d766224ae4ab6" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 42225 } } peers { permanent_uuid: "87d4ebb338564421816a43cffeb42715" member_type: VOTER last_known_addr { host: "127.15.113.189" port: 41115 } } peers { permanent_uuid: "70317072228d4a47877776b7fb4e21d1" member_type: VOTER last_known_addr { host: "127.15.113.188" port: 43669 } }
I20260809 22:53:11.910036 26947 sys_catalog.cc:455] T 00000000000000000000000000000000 P 70317072228d4a47877776b7fb4e21d1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "460ec5412d65461a811d766224ae4ab6" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 42225 } } peers { permanent_uuid: "87d4ebb338564421816a43cffeb42715" member_type: VOTER last_known_addr { host: "127.15.113.189" port: 41115 } } peers { permanent_uuid: "70317072228d4a47877776b7fb4e21d1" member_type: VOTER last_known_addr { host: "127.15.113.188" port: 43669 } } }
I20260809 22:53:11.910830 26947 sys_catalog.cc:458] T 00000000000000000000000000000000 P 70317072228d4a47877776b7fb4e21d1 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 22:53:11.911132 26938 sys_catalog.cc:564] T 00000000000000000000000000000000 P 70317072228d4a47877776b7fb4e21d1 [sys.catalog]: configured and running, proceeding with master startup.
W20260809 22:53:11.927763 26961 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 70317072228d4a47877776b7fb4e21d1: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 22:53:11.927990 26961 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 22:53:11.933284 26693 master_runner.cc:428] Detected that this master 70317072228d4a47877776b7fb4e21d1 is joining an existing cluster
I20260809 22:53:11.933390 26693 master_runner.cc:432] Initiating AddMaster RPC to add 127.15.113.188:43669
I20260809 22:53:11.946647 15814 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskItYPJR/build/asan/bin/kudu pid 25881
I20260809 22:53:11.970675 26943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:11.940329 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52218 (local address 127.15.113.190:42225)
0809 22:53:11.940532 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0809 22:53:11.940539 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:53:11.942835 (+ 2296us) server_negotiation.cc:407] Connection header received
0809 22:53:11.943152 (+ 317us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:53:11.943159 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:53:11.943249 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:53:11.943333 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:53:11.949065 (+ 5732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:11.950655 (+ 1590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:11.954106 (+ 3451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:11.955118 (+ 1012us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:11.955237 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:11.962093 (+ 6856us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:53:11.962100 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:53:11.962105 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:53:11.962148 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:53:11.966357 (+ 4209us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:53:11.967484 (+ 1127us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:53:11.967491 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:53:11.967494 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:53:11.967749 (+ 255us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:53:11.968416 (+ 667us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:53:11.968421 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:53:11.968426 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:53:11.968976 (+ 550us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:53:11.969146 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0809 22:53:11.969943 (+ 797us) server_negotiation.cc:299] Negotiation successful
0809 22:53:11.970186 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260809 22:53:11.972585 26971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:11.940988 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:42225 (local address 127.0.0.1:52218)
0809 22:53:11.942723 (+ 1735us) negotiation.cc:107] Waiting for socket to connect
0809 22:53:11.942745 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0809 22:53:11.942975 (+ 230us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:53:11.943580 (+ 605us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:53:11.943585 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:53:11.944674 (+ 1089us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:53:11.948816 (+ 4142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:53:11.948836 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:53:11.950913 (+ 2077us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:53:11.950920 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:53:11.953900 (+ 2980us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:53:11.953924 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:53:11.955723 (+ 1799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:53:11.955734 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:53:11.957153 (+ 1419us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:11.960046 (+ 2893us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:53:11.960076 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:53:11.961848 (+ 1772us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:53:11.966616 (+ 4768us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:53:11.966622 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:53:11.966627 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:53:11.967343 (+ 716us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:53:11.967897 (+ 554us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:53:11.967902 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:53:11.967906 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:53:11.968260 (+ 354us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:53:11.969227 (+ 967us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:53:11.969231 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:53:11.969636 (+ 405us) client_negotiation.cc:769] Sending connection context
0809 22:53:11.971190 (+ 1554us) client_negotiation.cc:240] Negotiation successful
0809 22:53:11.971636 (+ 446us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1516,"mutex_wait_us":25,"thread_start_us":1206,"threads_started":1}
Aug 09 22:53:11 dist-test-slave-ncp9 krb5kdc[25693](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315988, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.15.113.129@KRBTEST.COM
I20260809 22:53:11.985997 26972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:11.943129 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.188:43669 (local address 127.0.0.1:37802)
0809 22:53:11.965888 (+ 22759us) negotiation.cc:107] Waiting for socket to connect
0809 22:53:11.965952 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0809 22:53:11.966127 (+ 175us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:53:11.966627 (+ 500us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:53:11.966631 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:53:11.968085 (+ 1454us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:53:11.969096 (+ 1011us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:53:11.969111 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:53:11.970846 (+ 1735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:53:11.970851 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:53:11.973290 (+ 2439us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:53:11.973302 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:53:11.974741 (+ 1439us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:53:11.974750 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:53:11.975004 (+ 254us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:11.975663 (+ 659us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:53:11.975694 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:53:11.976689 (+ 995us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:53:11.982150 (+ 5461us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:53:11.982154 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:53:11.982157 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:53:11.982856 (+ 699us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:53:11.983288 (+ 432us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:53:11.983292 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:53:11.983296 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:53:11.983525 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:53:11.984389 (+ 864us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:53:11.984393 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:53:11.984563 (+ 170us) client_negotiation.cc:769] Sending connection context
0809 22:53:11.984796 (+ 233us) client_negotiation.cc:240] Negotiation successful
0809 22:53:11.985157 (+ 361us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":22496,"mutex_wait_us":1,"thread_start_us":7333,"threads_started":1}
I20260809 22:53:11.986981 26941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:11.942835 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37802 (local address 127.15.113.188:43669)
0809 22:53:11.943021 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0809 22:53:11.943028 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:53:11.966033 (+ 23005us) server_negotiation.cc:407] Connection header received
0809 22:53:11.966257 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:53:11.966262 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:53:11.966330 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:53:11.966420 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:53:11.969233 (+ 2813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:11.970607 (+ 1374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:11.973414 (+ 2807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:11.974098 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:11.974213 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:11.976935 (+ 2722us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:53:11.976943 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:53:11.976948 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:53:11.976992 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:53:11.982008 (+ 5016us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:53:11.982980 (+ 972us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:53:11.982985 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:53:11.982989 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:53:11.983158 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:53:11.983655 (+ 497us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:53:11.983659 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:53:11.983662 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:53:11.984168 (+ 506us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:53:11.984331 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0809 22:53:11.986295 (+ 1964us) server_negotiation.cc:299] Negotiation successful
0809 22:53:11.986521 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":34,"server-negotiator.queue_time_us":54}
I20260809 22:53:11.988731 26976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:11.948748 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39802 (local address 127.15.113.129:38043)
0809 22:53:11.958054 (+ 9306us) server_negotiation.cc:206] Beginning negotiation
0809 22:53:11.958064 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 22:53:11.958097 (+ 33us) server_negotiation.cc:407] Connection header received
0809 22:53:11.958169 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:53:11.958176 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:53:11.958410 (+ 234us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:53:11.958569 (+ 159us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:53:11.961141 (+ 2572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:11.962782 (+ 1641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:11.967542 (+ 4760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:11.968873 (+ 1331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:11.976292 (+ 7419us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:11.978010 (+ 1718us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:53:11.978022 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:53:11.978037 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:53:11.978069 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:53:11.983462 (+ 5393us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:53:11.984621 (+ 1159us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:53:11.984631 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:53:11.984635 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:53:11.984939 (+ 304us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:53:11.985594 (+ 655us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:53:11.985599 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:53:11.985601 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:53:11.986380 (+ 779us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:53:11.986821 (+ 441us) server_negotiation.cc:1035] Waiting for connection context
0809 22:53:11.987820 (+ 999us) server_negotiation.cc:299] Negotiation successful
0809 22:53:11.988094 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9030,"thread_start_us":9061,"threads_started":1}
I20260809 22:53:11.990124 26822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:11.940344 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50720 (local address 127.15.113.189:41115)
0809 22:53:11.940533 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0809 22:53:11.940539 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:53:11.950883 (+ 10344us) server_negotiation.cc:407] Connection header received
0809 22:53:11.951042 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:53:11.951048 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:53:11.951134 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:53:11.951222 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:53:11.954138 (+ 2916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:11.955653 (+ 1515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:11.969669 (+ 14016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:11.970807 (+ 1138us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:11.970909 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:11.980203 (+ 9294us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:53:11.980210 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:53:11.980215 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:53:11.980260 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:53:11.985076 (+ 4816us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:53:11.985979 (+ 903us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:53:11.985985 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:53:11.985988 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:53:11.986196 (+ 208us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:53:11.986689 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:53:11.986694 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:53:11.986697 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:53:11.987120 (+ 423us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:53:11.987249 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0809 22:53:11.987986 (+ 737us) server_negotiation.cc:299] Negotiation successful
0809 22:53:11.988321 (+ 335us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260809 22:53:11.990654 26087 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:53:11.991535 26973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:11.941254 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.189:41115 (local address 127.0.0.1:50720)
0809 22:53:11.950773 (+ 9519us) negotiation.cc:107] Waiting for socket to connect
0809 22:53:11.950791 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 22:53:11.950915 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:53:11.951408 (+ 493us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:53:11.951414 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:53:11.952927 (+ 1513us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:53:11.953945 (+ 1018us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:53:11.953960 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:53:11.955932 (+ 1972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:53:11.955938 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:53:11.968047 (+ 12109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:53:11.968062 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:53:11.971647 (+ 3585us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:53:11.971657 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:53:11.972034 (+ 377us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:11.978158 (+ 6124us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:53:11.978182 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:53:11.979942 (+ 1760us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:53:11.985238 (+ 5296us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:53:11.985243 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:53:11.985247 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:53:11.985827 (+ 580us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:53:11.986345 (+ 518us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:53:11.986349 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:53:11.986352 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:53:11.986556 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:53:11.987339 (+ 783us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:53:11.987343 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:53:11.987659 (+ 316us) client_negotiation.cc:769] Sending connection context
0809 22:53:11.990403 (+ 2744us) client_negotiation.cc:240] Negotiation successful
0809 22:53:11.990763 (+ 360us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9422,"thread_start_us":23915,"threads_started":1}
W20260809 22:53:12.014504 26861 tablet.cc:2369] T 00000000000000000000000000000000 P 70317072228d4a47877776b7fb4e21d1: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 09 22:53:12 dist-test-slave-ncp9 krb5kdc[25693](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315988, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.113.190@KRBTEST.COM for oryx/127.15.113.188@KRBTEST.COM
I20260809 22:53:12.040647 26978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:12.018128 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.188:43669 (local address 127.0.0.1:37810)
0809 22:53:12.019089 (+ 961us) negotiation.cc:107] Waiting for socket to connect
0809 22:53:12.019110 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0809 22:53:12.019257 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:53:12.019722 (+ 465us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:53:12.019727 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:53:12.020347 (+ 620us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:53:12.020884 (+ 537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:53:12.020925 (+ 41us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:53:12.022798 (+ 1873us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:53:12.022805 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:53:12.024478 (+ 1673us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:53:12.024495 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:53:12.026230 (+ 1735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:53:12.026236 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:53:12.026586 (+ 350us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:12.027475 (+ 889us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:53:12.027499 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:53:12.031052 (+ 3553us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:53:12.037091 (+ 6039us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:53:12.037097 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:53:12.037101 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:53:12.037781 (+ 680us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:53:12.038332 (+ 551us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:53:12.038337 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:53:12.038341 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:53:12.038602 (+ 261us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:53:12.039376 (+ 774us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:53:12.039382 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:53:12.039632 (+ 250us) client_negotiation.cc:769] Sending connection context
0809 22:53:12.039773 (+ 141us) client_negotiation.cc:240] Negotiation successful
0809 22:53:12.040105 (+ 332us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":835,"thread_start_us":702,"threads_started":1}
I20260809 22:53:12.041814 26941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:12.018514 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37810 (local address 127.15.113.188:43669)
0809 22:53:12.018714 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0809 22:53:12.018720 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:53:12.019202 (+ 482us) server_negotiation.cc:407] Connection header received
0809 22:53:12.019419 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:53:12.019425 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:53:12.019564 (+ 139us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:53:12.019663 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:53:12.021110 (+ 1447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:12.022629 (+ 1519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:12.024651 (+ 2022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:12.025829 (+ 1178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:12.026002 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:12.031432 (+ 5430us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:53:12.031467 (+ 35us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:53:12.031472 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:53:12.031512 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:53:12.036887 (+ 5375us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:53:12.037920 (+ 1033us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:53:12.037926 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:53:12.037930 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:53:12.038195 (+ 265us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:53:12.038732 (+ 537us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:53:12.038737 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:53:12.038741 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:53:12.039176 (+ 435us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:53:12.039315 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0809 22:53:12.041037 (+ 1722us) server_negotiation.cc:299] Negotiation successful
0809 22:53:12.041297 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260809 22:53:12.043933 25770 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.15.113.188:43669: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "70317072228d4a47877776b7fb4e21d1"
member_type: NON_VOTER
last_known_addr {
host: "127.15.113.188"
port: 43669
}
attrs {
promote: true
}
}
dest_uuid: "460ec5412d65461a811d766224ae4ab6"
cas_config_opid_index: 5
I20260809 22:53:12.049893 25770 consensus_queue.cc:237] T 00000000000000000000000000000000 P 460ec5412d65461a811d766224ae4ab6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "460ec5412d65461a811d766224ae4ab6" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 42225 } } peers { permanent_uuid: "87d4ebb338564421816a43cffeb42715" member_type: NON_VOTER last_known_addr { host: "127.15.113.189" port: 41115 } attrs { promote: true } } peers { permanent_uuid: "70317072228d4a47877776b7fb4e21d1" member_type: NON_VOTER last_known_addr { host: "127.15.113.188" port: 43669 } attrs { promote: true } }
I20260809 22:53:12.056819 26980 sys_catalog.cc:455] T 00000000000000000000000000000000 P 460ec5412d65461a811d766224ae4ab6 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "460ec5412d65461a811d766224ae4ab6" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "460ec5412d65461a811d766224ae4ab6" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 42225 } } peers { permanent_uuid: "87d4ebb338564421816a43cffeb42715" member_type: NON_VOTER last_known_addr { host: "127.15.113.189" port: 41115 } attrs { promote: true } } } pending_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "460ec5412d65461a811d766224ae4ab6" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 42225 } } peers { permanent_uuid: "87d4ebb338564421816a43cffeb42715" member_type: NON_VOTER last_known_addr { host: "127.15.113.189" port: 41115 } attrs { promote: true } } peers { permanent_uuid: "70317072228d4a47877776b7fb4e21d1" member_type: NON_VOTER last_known_addr { host: "127.15.113.188" port: 43669 } attrs { promote: true } } }
I20260809 22:53:12.057600 26980 sys_catalog.cc:458] T 00000000000000000000000000000000 P 460ec5412d65461a811d766224ae4ab6 [sys.catalog]: This master's current role is: LEADER
I20260809 22:53:12.063657 26895 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 70317072228d4a47877776b7fb4e21d1 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:53:12.069123 26895 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 70317072228d4a47877776b7fb4e21d1 [term 1 FOLLOWER]: Refusing update from remote peer 460ec5412d65461a811d766224ae4ab6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260809 22:53:12.061668 26984 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 460ec5412d65461a811d766224ae4ab6 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 70317072228d4a47877776b7fb4e21d1 (127.15.113.188) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "460ec5412d65461a811d766224ae4ab6" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 42225 } } peers { permanent_uuid: "87d4ebb338564421816a43cffeb42715" member_type: NON_VOTER last_known_addr { host: "127.15.113.189" port: 41115 } attrs { promote: true } } peers { permanent_uuid: "70317072228d4a47877776b7fb4e21d1" member_type: NON_VOTER last_known_addr { host: "127.15.113.188" port: 43669 } attrs { promote: true } } }
I20260809 22:53:12.070621 26763 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 87d4ebb338564421816a43cffeb42715 [term 1 LEARNER]: Refusing update from remote peer 460ec5412d65461a811d766224ae4ab6: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260809 22:53:12.071862 26986 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 460ec5412d65461a811d766224ae4ab6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "87d4ebb338564421816a43cffeb42715" member_type: NON_VOTER last_known_addr { host: "127.15.113.189" port: 41115 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260809 22:53:12.076017 26983 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 460ec5412d65461a811d766224ae4ab6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "70317072228d4a47877776b7fb4e21d1" member_type: NON_VOTER last_known_addr { host: "127.15.113.188" port: 43669 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.001s
I20260809 22:53:12.077355 26984 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.15.113.188:43669
I20260809 22:53:12.078047 26693 master.cc:537] Master@127.15.113.188:43669 shutting down...
I20260809 22:53:12.079813 26984 sys_catalog.cc:455] T 00000000000000000000000000000000 P 460ec5412d65461a811d766224ae4ab6 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "460ec5412d65461a811d766224ae4ab6" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "460ec5412d65461a811d766224ae4ab6" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 42225 } } peers { permanent_uuid: "87d4ebb338564421816a43cffeb42715" member_type: NON_VOTER last_known_addr { host: "127.15.113.189" port: 41115 } attrs { promote: true } } peers { permanent_uuid: "70317072228d4a47877776b7fb4e21d1" member_type: NON_VOTER last_known_addr { host: "127.15.113.188" port: 43669 } attrs { promote: true } } }
I20260809 22:53:12.080202 26984 sys_catalog.cc:458] T 00000000000000000000000000000000 P 460ec5412d65461a811d766224ae4ab6 [sys.catalog]: This master's current role is: LEADER
I20260809 22:53:12.082667 26763 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 87d4ebb338564421816a43cffeb42715 [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 70317072228d4a47877776b7fb4e21d1 (127.15.113.188) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "460ec5412d65461a811d766224ae4ab6" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 42225 } } peers { permanent_uuid: "87d4ebb338564421816a43cffeb42715" member_type: NON_VOTER last_known_addr { host: "127.15.113.189" port: 41115 } attrs { promote: true } } peers { permanent_uuid: "70317072228d4a47877776b7fb4e21d1" member_type: NON_VOTER last_known_addr { host: "127.15.113.188" port: 43669 } attrs { promote: true } } }
I20260809 22:53:12.098309 26986 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 460ec5412d65461a811d766224ae4ab6: attempting to promote NON_VOTER 87d4ebb338564421816a43cffeb42715 to VOTER
I20260809 22:53:12.097858 26993 sys_catalog.cc:455] T 00000000000000000000000000000000 P 87d4ebb338564421816a43cffeb42715 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "460ec5412d65461a811d766224ae4ab6" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "460ec5412d65461a811d766224ae4ab6" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 42225 } } peers { permanent_uuid: "87d4ebb338564421816a43cffeb42715" member_type: NON_VOTER last_known_addr { host: "127.15.113.189" port: 41115 } attrs { promote: true } } peers { permanent_uuid: "70317072228d4a47877776b7fb4e21d1" member_type: NON_VOTER last_known_addr { host: "127.15.113.188" port: 43669 } attrs { promote: true } } }
I20260809 22:53:12.098798 26993 sys_catalog.cc:458] T 00000000000000000000000000000000 P 87d4ebb338564421816a43cffeb42715 [sys.catalog]: This master's current role is: LEARNER
I20260809 22:53:12.099548 26986 consensus_queue.cc:237] T 00000000000000000000000000000000 P 460ec5412d65461a811d766224ae4ab6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "460ec5412d65461a811d766224ae4ab6" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 42225 } } peers { permanent_uuid: "87d4ebb338564421816a43cffeb42715" member_type: VOTER last_known_addr { host: "127.15.113.189" port: 41115 } attrs { promote: false } } peers { permanent_uuid: "70317072228d4a47877776b7fb4e21d1" member_type: NON_VOTER last_known_addr { host: "127.15.113.188" port: 43669 } attrs { promote: true } }
I20260809 22:53:12.101024 26693 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 70317072228d4a47877776b7fb4e21d1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 22:53:12.101431 26693 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 70317072228d4a47877776b7fb4e21d1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 22:53:12.101639 26693 tablet_replica.cc:330] T 00000000000000000000000000000000 P 70317072228d4a47877776b7fb4e21d1: stopping tablet replica
I20260809 22:53:12.103806 26762 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 87d4ebb338564421816a43cffeb42715 [term 1 LEARNER]: Refusing update from remote peer 460ec5412d65461a811d766224ae4ab6: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260809 22:53:12.104555 26983 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 460ec5412d65461a811d766224ae4ab6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "87d4ebb338564421816a43cffeb42715" member_type: VOTER last_known_addr { host: "127.15.113.189" port: 41115 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
W20260809 22:53:12.105296 25745 consensus_peers.cc:487] T 00000000000000000000000000000000 P 460ec5412d65461a811d766224ae4ab6 -> Peer 70317072228d4a47877776b7fb4e21d1 (127.15.113.188:43669): Couldn't send request to peer 70317072228d4a47877776b7fb4e21d1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260809 22:53:12.107378 26983 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 460ec5412d65461a811d766224ae4ab6 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 87d4ebb338564421816a43cffeb42715 (127.15.113.189) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "460ec5412d65461a811d766224ae4ab6" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 42225 } } peers { permanent_uuid: "87d4ebb338564421816a43cffeb42715" member_type: VOTER last_known_addr { host: "127.15.113.189" port: 41115 } attrs { promote: false } } peers { permanent_uuid: "70317072228d4a47877776b7fb4e21d1" member_type: NON_VOTER last_known_addr { host: "127.15.113.188" port: 43669 } attrs { promote: true } } }
I20260809 22:53:12.107853 26762 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 87d4ebb338564421816a43cffeb42715 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 87d4ebb338564421816a43cffeb42715 (127.15.113.189) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "460ec5412d65461a811d766224ae4ab6" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 42225 } } peers { permanent_uuid: "87d4ebb338564421816a43cffeb42715" member_type: VOTER last_known_addr { host: "127.15.113.189" port: 41115 } attrs { promote: false } } peers { permanent_uuid: "70317072228d4a47877776b7fb4e21d1" member_type: NON_VOTER last_known_addr { host: "127.15.113.188" port: 43669 } attrs { promote: true } } }
I20260809 22:53:12.111716 26993 sys_catalog.cc:455] T 00000000000000000000000000000000 P 87d4ebb338564421816a43cffeb42715 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "460ec5412d65461a811d766224ae4ab6" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "460ec5412d65461a811d766224ae4ab6" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 42225 } } peers { permanent_uuid: "87d4ebb338564421816a43cffeb42715" member_type: VOTER last_known_addr { host: "127.15.113.189" port: 41115 } attrs { promote: false } } peers { permanent_uuid: "70317072228d4a47877776b7fb4e21d1" member_type: NON_VOTER last_known_addr { host: "127.15.113.188" port: 43669 } attrs { promote: true } } }
I20260809 22:53:12.112113 26993 sys_catalog.cc:458] T 00000000000000000000000000000000 P 87d4ebb338564421816a43cffeb42715 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 22:53:12.112751 26983 sys_catalog.cc:455] T 00000000000000000000000000000000 P 460ec5412d65461a811d766224ae4ab6 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "460ec5412d65461a811d766224ae4ab6" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "460ec5412d65461a811d766224ae4ab6" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 42225 } } peers { permanent_uuid: "87d4ebb338564421816a43cffeb42715" member_type: VOTER last_known_addr { host: "127.15.113.189" port: 41115 } attrs { promote: false } } peers { permanent_uuid: "70317072228d4a47877776b7fb4e21d1" member_type: NON_VOTER last_known_addr { host: "127.15.113.188" port: 43669 } attrs { promote: true } } }
I20260809 22:53:12.113260 26983 sys_catalog.cc:458] T 00000000000000000000000000000000 P 460ec5412d65461a811d766224ae4ab6 [sys.catalog]: This master's current role is: LEADER
I20260809 22:53:12.112728 26986 sys_catalog.cc:455] T 00000000000000000000000000000000 P 460ec5412d65461a811d766224ae4ab6 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "460ec5412d65461a811d766224ae4ab6" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "460ec5412d65461a811d766224ae4ab6" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 42225 } } peers { permanent_uuid: "87d4ebb338564421816a43cffeb42715" member_type: VOTER last_known_addr { host: "127.15.113.189" port: 41115 } attrs { promote: false } } peers { permanent_uuid: "70317072228d4a47877776b7fb4e21d1" member_type: NON_VOTER last_known_addr { host: "127.15.113.188" port: 43669 } attrs { promote: true } } }
I20260809 22:53:12.113700 26986 sys_catalog.cc:458] T 00000000000000000000000000000000 P 460ec5412d65461a811d766224ae4ab6 [sys.catalog]: This master's current role is: LEADER
I20260809 22:53:12.118556 26693 master.cc:552] Master@127.15.113.188:43669 shutdown complete.
I20260809 22:53:12.118651 26693 master_runner.cc:305] Clearing existing system tablet
I20260809 22:53:12.119254 26693 fs_manager.cc:359] Metadata directory not provided
I20260809 22:53:12.119496 26693 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:53:12.123895 26693 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.001s sys 0.005s
I20260809 22:53:12.126541 26996 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:53:12.128109 26693 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315935617919-15814-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:53:12.128180 15814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskItYPJR/build/asan/bin/kudu with pid 25881
I20260809 22:53:12.128353 26693 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.001s
I20260809 22:53:12.128473 26693 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315935617919-15814-0/minicluster-data/master-2/data,/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315935617919-15814-0/minicluster-data/master-2/wal
uuid: "70317072228d4a47877776b7fb4e21d1"
format_stamp: "Formatted at 2026-08-09 22:53:11 on dist-test-slave-ncp9"
server_key: "cf4e1571e35110a74dfd8be895685a4e"
server_key_iv: "173a607b5f31669f6c6495b1906827a1"
server_key_version: "encryptionkey@0"
I20260809 22:53:12.132838 26693 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 70317072228d4a47877776b7fb4e21d1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 22:53:12.141055 26693 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 70317072228d4a47877776b7fb4e21d1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260809 22:53:12.141299 26693 log.cc:1196] T 00000000000000000000000000000000 P 70317072228d4a47877776b7fb4e21d1: Deleting WAL directory at /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315935617919-15814-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260809 22:53:12.142237 26693 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 70317072228d4a47877776b7fb4e21d1: Deleting consensus metadata
I20260809 22:53:12.142627 26693 master_runner.cc:315] Copying system tablet from 127.15.113.190:42225
I20260809 22:53:12.146667 26693 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 70317072228d4a47877776b7fb4e21d1: tablet copy: Beginning tablet copy session from remote peer at address 127.15.113.190:42225
I20260809 22:53:12.163115 26943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:12.147500 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52224 (local address 127.15.113.190:42225)
0809 22:53:12.149026 (+ 1526us) server_negotiation.cc:206] Beginning negotiation
0809 22:53:12.149030 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 22:53:12.149047 (+ 17us) server_negotiation.cc:407] Connection header received
0809 22:53:12.149116 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:53:12.149119 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:53:12.149180 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:53:12.149272 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:53:12.150300 (+ 1028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:12.151441 (+ 1141us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:12.153329 (+ 1888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:12.153919 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:12.154023 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:12.156587 (+ 2564us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:53:12.156594 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:53:12.156599 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:53:12.156640 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:53:12.159984 (+ 3344us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:53:12.160770 (+ 786us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:53:12.160776 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:53:12.160779 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:53:12.160965 (+ 186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:53:12.161409 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:53:12.161414 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:53:12.161417 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:53:12.161740 (+ 323us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:53:12.161851 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0809 22:53:12.162486 (+ 635us) server_negotiation.cc:299] Negotiation successful
0809 22:53:12.162715 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1398}
I20260809 22:53:12.164024 27006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:12.147477 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:42225 (local address 127.0.0.1:52224)
0809 22:53:12.148038 (+ 561us) negotiation.cc:107] Waiting for socket to connect
0809 22:53:12.148058 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 22:53:12.148201 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:53:12.149325 (+ 1124us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:53:12.149330 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:53:12.149857 (+ 527us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:53:12.150121 (+ 264us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:53:12.150134 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:53:12.151647 (+ 1513us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:53:12.151652 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:53:12.153154 (+ 1502us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:53:12.153171 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:53:12.154292 (+ 1121us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:53:12.154297 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:53:12.154463 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:12.155259 (+ 796us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:53:12.155281 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:53:12.156358 (+ 1077us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:53:12.160159 (+ 3801us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:53:12.160163 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:53:12.160168 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:53:12.160636 (+ 468us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:53:12.161109 (+ 473us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:53:12.161113 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:53:12.161116 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:53:12.161287 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:53:12.161941 (+ 654us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:53:12.161945 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:53:12.162144 (+ 199us) client_negotiation.cc:769] Sending connection context
0809 22:53:12.163383 (+ 1239us) client_negotiation.cc:240] Negotiation successful
0809 22:53:12.163695 (+ 312us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":398,"thread_start_us":240,"threads_started":1}
I20260809 22:53:12.164678 25806 tablet_copy_service.cc:140] P 460ec5412d65461a811d766224ae4ab6: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 70317072228d4a47877776b7fb4e21d1 ({username='oryx', principal='oryx/127.15.113.188@KRBTEST.COM'} at 127.0.0.1:52224)
I20260809 22:53:12.164696 15814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskItYPJR/build/asan/bin/kudu
/tmp/dist-test-taskItYPJR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315935617919-15814-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315935617919-15814-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315935617919-15814-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315935617919-15814-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.113.129:38043
--local_ip_for_outbound_sockets=127.15.113.129
--tserver_master_addrs=127.15.113.190:42225
--webserver_port=42149
--webserver_interface=127.15.113.129
--keytab_file=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315935617919-15814-0/krb5kdc/oryx_127.15.113.129.keytab
--principal=oryx/127.15.113.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.113.148:37185
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.15.113.190:42225,127.15.113.189:41115
--tserver_master_addrs=127.15.113.190:42225,127.15.113.189:41115,127.15.113.188:43669 with env {KRB5CCNAME=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315935617919-15814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315935617919-15814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315935617919-15814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 22:53:12.164846 25806 tablet_copy_service.cc:161] P 460ec5412d65461a811d766224ae4ab6: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 70317072228d4a47877776b7fb4e21d1 at {username='oryx', principal='oryx/127.15.113.188@KRBTEST.COM'} at 127.0.0.1:52224: session id = 70317072228d4a47877776b7fb4e21d1-00000000000000000000000000000000
I20260809 22:53:12.167357 25806 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 460ec5412d65461a811d766224ae4ab6: Tablet Copy: opened 0 blocks and 1 log segments
I20260809 22:53:12.168988 26693 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260809 22:53:12.174093 26693 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 70317072228d4a47877776b7fb4e21d1: tablet copy: Starting download of 0 data blocks...
I20260809 22:53:12.174381 26693 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 70317072228d4a47877776b7fb4e21d1: tablet copy: Starting download of 1 WAL segments...
I20260809 22:53:12.177518 26693 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 70317072228d4a47877776b7fb4e21d1: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260809 22:53:12.181560 25806 tablet_copy_service.cc:342] P 460ec5412d65461a811d766224ae4ab6: Request end of tablet copy session 70317072228d4a47877776b7fb4e21d1-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.15.113.188@KRBTEST.COM'} at 127.0.0.1:52224
I20260809 22:53:12.181702 25806 tablet_copy_service.cc:433] P 460ec5412d65461a811d766224ae4ab6: ending tablet copy session 70317072228d4a47877776b7fb4e21d1-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 70317072228d4a47877776b7fb4e21d1
I20260809 22:53:12.194003 26693 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:53:12.196573 27009 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:53:12.197970 27013 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:53:12.211095 27016 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:53:12.214223 26693 server_base.cc:1034] running on GCE node
I20260809 22:53:12.214572 26693 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:53:12.215224 26693 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:53:12.216413 26693 hybrid_clock.cc:648] HybridClock initialized: now 1786315992216380 us; error 23 us; skew 500 ppm
I20260809 22:53:12.216657 26693 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:53:12 dist-test-slave-ncp9 krb5kdc[25693](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315992, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.113.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:53:12.218895 26693 init.cc:377] Logged in from keytab as oryx/127.15.113.188@KRBTEST.COM (short username oryx)
I20260809 22:53:12.222162 26693 webserver.cc:460] Webserver started at http://127.15.113.188:39167/ using document root <none> and password file <none>
I20260809 22:53:12.222510 26693 fs_manager.cc:359] Metadata directory not provided
I20260809 22:53:12.222623 26693 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:53:12.225703 26693 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.000s sys 0.003s
I20260809 22:53:12.227757 27030 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:53:12.228215 26693 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.002s
I20260809 22:53:12.228339 26693 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315935617919-15814-0/minicluster-data/master-2/data,/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315935617919-15814-0/minicluster-data/master-2/wal
uuid: "70317072228d4a47877776b7fb4e21d1"
format_stamp: "Formatted at 2026-08-09 22:53:11 on dist-test-slave-ncp9"
server_key: "cf4e1571e35110a74dfd8be895685a4e"
server_key_iv: "173a607b5f31669f6c6495b1906827a1"
server_key_version: "encryptionkey@0"
I20260809 22:53:12.228541 26693 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315935617919-15814-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315935617919-15814-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315935617919-15814-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:53:12.258735 26693 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:53:12.262720 26693 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:53:12.324677 26693 rpc_server.cc:307] RPC server started. Bound to: 127.15.113.188:43669
I20260809 22:53:12.324838 27107 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.113.188:43669 every 8 connection(s)
I20260809 22:53:12.325757 26693 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315935617919-15814-0/minicluster-data/master-2/data/info.pb
I20260809 22:53:12.331327 27109 sys_catalog.cc:263] Verifying existing consensus state
I20260809 22:53:12.335538 27109 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 70317072228d4a47877776b7fb4e21d1: Bootstrap starting.
I20260809 22:53:12.354175 27109 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 70317072228d4a47877776b7fb4e21d1: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 22:53:12.354696 27109 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 70317072228d4a47877776b7fb4e21d1: Bootstrap complete.
I20260809 22:53:12.355854 27109 raft_consensus.cc:348] T 00000000000000000000000000000000 P 70317072228d4a47877776b7fb4e21d1 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "460ec5412d65461a811d766224ae4ab6" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 42225 } } peers { permanent_uuid: "87d4ebb338564421816a43cffeb42715" member_type: VOTER last_known_addr { host: "127.15.113.189" port: 41115 } attrs { promote: false } } peers { permanent_uuid: "70317072228d4a47877776b7fb4e21d1" member_type: NON_VOTER last_known_addr { host: "127.15.113.188" port: 43669 } attrs { promote: true } }
I20260809 22:53:12.356208 27109 raft_consensus.cc:729] T 00000000000000000000000000000000 P 70317072228d4a47877776b7fb4e21d1 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 70317072228d4a47877776b7fb4e21d1, State: Initialized, Role: LEARNER
I20260809 22:53:12.356492 27109 consensus_queue.cc:260] T 00000000000000000000000000000000 P 70317072228d4a47877776b7fb4e21d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "460ec5412d65461a811d766224ae4ab6" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 42225 } } peers { permanent_uuid: "87d4ebb338564421816a43cffeb42715" member_type: VOTER last_known_addr { host: "127.15.113.189" port: 41115 } attrs { promote: false } } peers { permanent_uuid: "70317072228d4a47877776b7fb4e21d1" member_type: NON_VOTER last_known_addr { host: "127.15.113.188" port: 43669 } attrs { promote: true } }
I20260809 22:53:12.357409 27114 sys_catalog.cc:455] T 00000000000000000000000000000000 P 70317072228d4a47877776b7fb4e21d1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "460ec5412d65461a811d766224ae4ab6" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 42225 } } peers { permanent_uuid: "87d4ebb338564421816a43cffeb42715" member_type: VOTER last_known_addr { host: "127.15.113.189" port: 41115 } attrs { promote: false } } peers { permanent_uuid: "70317072228d4a47877776b7fb4e21d1" member_type: NON_VOTER last_known_addr { host: "127.15.113.188" port: 43669 } attrs { promote: true } } }
I20260809 22:53:12.357774 27109 sys_catalog.cc:564] T 00000000000000000000000000000000 P 70317072228d4a47877776b7fb4e21d1 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 22:53:12.357946 27114 sys_catalog.cc:458] T 00000000000000000000000000000000 P 70317072228d4a47877776b7fb4e21d1 [sys.catalog]: This master's current role is: LEARNER
I20260809 22:53:12.366837 27131 catalog_manager.cc:1259] Loaded cluster ID: 2dcfd0b4c82745d78a39e01701586553
I20260809 22:53:12.366983 27131 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 70317072228d4a47877776b7fb4e21d1: loading cluster ID for follower catalog manager: success
I20260809 22:53:12.370669 27131 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 70317072228d4a47877776b7fb4e21d1: acquiring CA information for follower catalog manager: success
I20260809 22:53:12.373247 27131 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 70317072228d4a47877776b7fb4e21d1: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260809 22:53:12.611871 27007 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:53:12.612277 27007 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:53:12.612564 27007 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:53:12.625571 27007 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:53:12.625708 27007 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:53:12.625907 27007 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.113.129
I20260809 22:53:12.641764 27007 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.113.148:37185
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315935617919-15814-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315935617919-15814-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315935617919-15814-0/krb5kdc/oryx_127.15.113.129.keytab
--principal=oryx/127.15.113.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.113.129:38043
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315935617919-15814-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.113.129
--webserver_port=42149
--tserver_master_addrs=127.15.113.190:42225,127.15.113.189:41115,127.15.113.188:43669
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.113.129
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315935617919-15814-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:53:12.643791 27007 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:53:12.645834 27007 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:53:12.662501 27141 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:53:12.662526 27142 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:53:12.663839 27148 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:53:12.664212 27007 server_base.cc:1034] running on GCE node
I20260809 22:53:12.664831 27007 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:53:12.665786 27007 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:53:12.667001 27007 hybrid_clock.cc:648] HybridClock initialized: now 1786315992666872 us; error 106 us; skew 500 ppm
I20260809 22:53:12.667291 27007 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:53:12 dist-test-slave-ncp9 krb5kdc[25693](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315992, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.113.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:53:12.671340 27007 init.cc:377] Logged in from keytab as oryx/127.15.113.129@KRBTEST.COM (short username oryx)
I20260809 22:53:12.672849 27007 webserver.cc:460] Webserver started at http://127.15.113.129:42149/ using document root <none> and password file <none>
I20260809 22:53:12.673653 27007 fs_manager.cc:359] Metadata directory not provided
I20260809 22:53:12.673806 27007 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:53:12.679446 27007 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260809 22:53:12.682360 27157 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:53:12.683940 27007 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 22:53:12.684085 27007 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315935617919-15814-0/minicluster-data/ts-0/data,/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315935617919-15814-0/minicluster-data/ts-0/wal
uuid: "b8e298532adf4449955b1b5dc49552c2"
format_stamp: "Formatted at 2026-08-09 22:53:09 on dist-test-slave-ncp9"
server_key: "889049b1e44c67a336a386c60c7a0163"
server_key_iv: "55b58844f827b812997ddaeb0c80a78a"
server_key_version: "encryptionkey@0"
I20260809 22:53:12.685045 27007 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315935617919-15814-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315935617919-15814-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315935617919-15814-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:53:12.696138 27159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:12.683208 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37812 (local address 127.15.113.188:43669)
0809 22:53:12.683747 (+ 539us) server_negotiation.cc:206] Beginning negotiation
0809 22:53:12.683753 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:53:12.689991 (+ 6238us) server_negotiation.cc:407] Connection header received
0809 22:53:12.690207 (+ 216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:53:12.690213 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:53:12.690317 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:53:12.690443 (+ 126us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:53:12.690870 (+ 427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:12.691941 (+ 1071us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:12.694087 (+ 2146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:12.695258 (+ 1171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:12.695427 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:12.695555 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0809 22:53:12.695764 (+ 209us) server_negotiation.cc:299] Negotiation successful
0809 22:53:12.695858 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":395,"thread_start_us":236,"threads_started":1}
I20260809 22:53:12.696142 27158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:12.683091 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.188:43669 (local address 127.0.0.1:37812)
0809 22:53:12.689862 (+ 6771us) negotiation.cc:107] Waiting for socket to connect
0809 22:53:12.689884 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0809 22:53:12.690038 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:53:12.690474 (+ 436us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:53:12.690479 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:53:12.690510 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:53:12.690719 (+ 209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:53:12.690731 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:53:12.692083 (+ 1352us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:53:12.692088 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:53:12.693929 (+ 1841us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:53:12.693944 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:53:12.695418 (+ 1474us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:53:12.695423 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:53:12.695617 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:12.695650 (+ 33us) client_negotiation.cc:769] Sending connection context
0809 22:53:12.695729 (+ 79us) client_negotiation.cc:240] Negotiation successful
0809 22:53:12.695858 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6637,"thread_start_us":6538,"threads_started":1}
I20260809 22:53:12.697114 27070 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 70317072228d4a47877776b7fb4e21d1 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20260809 22:53:12.697661 27114 sys_catalog.cc:455] T 00000000000000000000000000000000 P 70317072228d4a47877776b7fb4e21d1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 460ec5412d65461a811d766224ae4ab6. Latest consensus state: current_term: 1 leader_uuid: "460ec5412d65461a811d766224ae4ab6" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "460ec5412d65461a811d766224ae4ab6" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 42225 } } peers { permanent_uuid: "87d4ebb338564421816a43cffeb42715" member_type: VOTER last_known_addr { host: "127.15.113.189" port: 41115 } attrs { promote: false } } peers { permanent_uuid: "70317072228d4a47877776b7fb4e21d1" member_type: NON_VOTER last_known_addr { host: "127.15.113.188" port: 43669 } attrs { promote: true } } }
I20260809 22:53:12.697957 27114 sys_catalog.cc:458] T 00000000000000000000000000000000 P 70317072228d4a47877776b7fb4e21d1 [sys.catalog]: This master's current role is: LEARNER
I20260809 22:53:12.700629 27007 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:53:12.705269 27007 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:53:12.705564 27007 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:53:12.706997 27007 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:53:12.708545 27007 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:53:12.708653 27007 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:53:12.708796 27007 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:53:12.708940 27007 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:53:12.778487 27007 rpc_server.cc:307] RPC server started. Bound to: 127.15.113.129:38043
I20260809 22:53:12.778683 27344 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.113.129:38043 every 8 connection(s)
I20260809 22:53:12.780457 27007 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315935617919-15814-0/minicluster-data/ts-0/data/info.pb
I20260809 22:53:12.781467 15814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskItYPJR/build/asan/bin/kudu as pid 27007
I20260809 22:53:12.781883 15814 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskItYPJR/build/asan/bin/kudu pid 26101
Aug 09 22:53:12 dist-test-slave-ncp9 krb5kdc[25693](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315992, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.113.129@KRBTEST.COM for oryx/127.15.113.188@KRBTEST.COM
Aug 09 22:53:12 dist-test-slave-ncp9 krb5kdc[25693](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315988, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.15.113.130@KRBTEST.COM
Aug 09 22:53:12 dist-test-slave-ncp9 krb5kdc[25693](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315992, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.113.129@KRBTEST.COM for oryx/127.15.113.190@KRBTEST.COM
I20260809 22:53:12.846400 27159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:12.786160 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.113.129:33013 (local address 127.15.113.188:43669)
0809 22:53:12.786390 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0809 22:53:12.786397 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:53:12.795313 (+ 8916us) server_negotiation.cc:407] Connection header received
0809 22:53:12.805234 (+ 9921us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:53:12.805242 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:53:12.805344 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:53:12.805466 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:53:12.808604 (+ 3138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:12.810084 (+ 1480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:12.814266 (+ 4182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:12.815263 (+ 997us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:12.815372 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:12.833304 (+ 17932us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:53:12.833315 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:53:12.833321 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:53:12.833380 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:53:12.840394 (+ 7014us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:53:12.842062 (+ 1668us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:53:12.842070 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:53:12.842074 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:53:12.842316 (+ 242us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:53:12.843523 (+ 1207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:53:12.843529 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:53:12.843533 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:53:12.843975 (+ 442us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:53:12.844146 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0809 22:53:12.845444 (+ 1298us) server_negotiation.cc:299] Negotiation successful
0809 22:53:12.845740 (+ 296us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
I20260809 22:53:12.850147 27358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:12.784928 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.188:43669 (local address 127.15.113.129:33013)
0809 22:53:12.795109 (+ 10181us) negotiation.cc:107] Waiting for socket to connect
0809 22:53:12.795164 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0809 22:53:12.805034 (+ 9870us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:53:12.806008 (+ 974us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:53:12.806022 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:53:12.807180 (+ 1158us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:53:12.808322 (+ 1142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:53:12.808359 (+ 37us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:53:12.810386 (+ 2027us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:53:12.810392 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:53:12.814081 (+ 3689us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:53:12.814098 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:53:12.815954 (+ 1856us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:53:12.815963 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:53:12.816395 (+ 432us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:12.823126 (+ 6731us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:53:12.823220 (+ 94us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:53:12.831145 (+ 7925us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:53:12.841042 (+ 9897us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:53:12.841058 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:53:12.841079 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:53:12.841866 (+ 787us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:53:12.843091 (+ 1225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:53:12.843094 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:53:12.843097 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:53:12.843376 (+ 279us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:53:12.844242 (+ 866us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:53:12.844251 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:53:12.844830 (+ 579us) client_negotiation.cc:769] Sending connection context
0809 22:53:12.847373 (+ 2543us) client_negotiation.cc:240] Negotiation successful
0809 22:53:12.848147 (+ 774us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9692,"thread_start_us":10328,"threads_started":1}
I20260809 22:53:12.852854 27346 heartbeater.cc:347] Connected to a master server at 127.15.113.188:43669
I20260809 22:53:12.853317 27346 heartbeater.cc:464] Registering TS with master...
I20260809 22:53:12.854612 27346 heartbeater.cc:511] Master 127.15.113.188:43669 requested a full tablet report, sending...
I20260809 22:53:12.856012 27357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:12.783319 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47816 (local address 127.15.113.130:37171)
0809 22:53:12.811482 (+ 28163us) server_negotiation.cc:206] Beginning negotiation
0809 22:53:12.811538 (+ 56us) server_negotiation.cc:399] Waiting for connection header
0809 22:53:12.811789 (+ 251us) server_negotiation.cc:407] Connection header received
0809 22:53:12.811868 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:53:12.811876 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:53:12.812152 (+ 276us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:53:12.815790 (+ 3638us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:53:12.816205 (+ 415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:12.818855 (+ 2650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:12.821356 (+ 2501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:12.827965 (+ 6609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:12.834195 (+ 6230us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:12.838241 (+ 4046us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:53:12.838259 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:53:12.838278 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:53:12.838323 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:53:12.844311 (+ 5988us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:53:12.848985 (+ 4674us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:53:12.848994 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:53:12.848997 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:53:12.849165 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:53:12.853521 (+ 4356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:53:12.853525 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:53:12.853527 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:53:12.853994 (+ 467us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:53:12.854396 (+ 402us) server_negotiation.cc:1035] Waiting for connection context
0809 22:53:12.855331 (+ 935us) server_negotiation.cc:299] Negotiation successful
0809 22:53:12.855620 (+ 289us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27683,"thread_start_us":27355,"threads_started":1}
I20260809 22:53:12.857565 26292 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:53:12.858282 27057 ts_manager.cc:194] Registered new tserver with Master: b8e298532adf4449955b1b5dc49552c2 (127.15.113.129:38043)
I20260809 22:53:12.859756 27366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:12.796691 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.113.129:38321 (local address 127.15.113.190:42225)
0809 22:53:12.809610 (+ 12919us) server_negotiation.cc:206] Beginning negotiation
0809 22:53:12.809617 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:53:12.809639 (+ 22us) server_negotiation.cc:407] Connection header received
0809 22:53:12.809706 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:53:12.809713 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:53:12.809799 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:53:12.811093 (+ 1294us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:53:12.814272 (+ 3179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:12.816262 (+ 1990us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:12.827107 (+ 10845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:12.838238 (+ 11131us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:12.838386 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:12.845263 (+ 6877us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:53:12.845275 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:53:12.845281 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:53:12.845326 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:53:12.851262 (+ 5936us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:53:12.852303 (+ 1041us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:53:12.852310 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:53:12.852313 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:53:12.852579 (+ 266us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:53:12.853104 (+ 525us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:53:12.853109 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:53:12.853112 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:53:12.853661 (+ 549us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:53:12.853851 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0809 22:53:12.858843 (+ 4992us) server_negotiation.cc:299] Negotiation successful
0809 22:53:12.859139 (+ 296us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12762,"thread_start_us":12616,"threads_started":1}
I20260809 22:53:12.859810 27364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:12.796603 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:42225 (local address 127.15.113.129:38321)
0809 22:53:12.804361 (+ 7758us) negotiation.cc:107] Waiting for socket to connect
0809 22:53:12.804380 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 22:53:12.804504 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:53:12.811111 (+ 6607us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:53:12.811116 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:53:12.813024 (+ 1908us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:53:12.814023 (+ 999us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:53:12.814041 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:53:12.824445 (+ 10404us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:53:12.824451 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:53:12.826820 (+ 2369us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:53:12.826838 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:53:12.838566 (+ 11728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:53:12.838577 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:53:12.838811 (+ 234us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:12.839702 (+ 891us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:53:12.839733 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:53:12.844816 (+ 5083us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:53:12.851438 (+ 6622us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:53:12.851443 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:53:12.851447 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:53:12.852150 (+ 703us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:53:12.852718 (+ 568us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:53:12.852725 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:53:12.852728 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:53:12.852955 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:53:12.853825 (+ 870us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:53:12.853830 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:53:12.854072 (+ 242us) client_negotiation.cc:769] Sending connection context
0809 22:53:12.854233 (+ 161us) client_negotiation.cc:240] Negotiation successful
0809 22:53:12.854604 (+ 371us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7617,"thread_start_us":8229,"threads_started":1}
I20260809 22:53:12.861086 27353 heartbeater.cc:347] Connected to a master server at 127.15.113.190:42225
I20260809 22:53:12.861212 27353 heartbeater.cc:464] Registering TS with master...
I20260809 22:53:12.861832 27353 heartbeater.cc:511] Master 127.15.113.190:42225 requested a full tablet report, sending...
I20260809 22:53:12.863318 25770 ts_manager.cc:194] Re-registered known tserver with Master: b8e298532adf4449955b1b5dc49552c2 (127.15.113.129:38043)
I20260809 22:53:12.864445 25770 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.15.113.129@KRBTEST.COM'} at 127.15.113.129:38321
Aug 09 22:53:12 dist-test-slave-ncp9 krb5kdc[25693](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315992, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.113.129@KRBTEST.COM for oryx/127.15.113.189@KRBTEST.COM
I20260809 22:53:12.875633 27370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:12.817583 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.113.129:59211 (local address 127.15.113.189:41115)
0809 22:53:12.842078 (+ 24495us) server_negotiation.cc:206] Beginning negotiation
0809 22:53:12.842086 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 22:53:12.842107 (+ 21us) server_negotiation.cc:407] Connection header received
0809 22:53:12.842171 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:53:12.842176 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:53:12.842295 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:53:12.842417 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:53:12.849330 (+ 6913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:12.851085 (+ 1755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:12.856271 (+ 5186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:12.859347 (+ 3076us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:12.859445 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:12.868737 (+ 9292us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:53:12.868744 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:53:12.868747 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:53:12.868779 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:53:12.872576 (+ 3797us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:53:12.873400 (+ 824us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:53:12.873404 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:53:12.873407 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:53:12.873640 (+ 233us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:53:12.874183 (+ 543us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:53:12.874186 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:53:12.874188 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:53:12.874474 (+ 286us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:53:12.874576 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0809 22:53:12.875083 (+ 507us) server_negotiation.cc:299] Negotiation successful
0809 22:53:12.875238 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24289,"thread_start_us":24991,"threads_started":1}
I20260809 22:53:12.875946 27372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:12.817252 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.189:41115 (local address 127.15.113.129:59211)
0809 22:53:12.835397 (+ 18145us) negotiation.cc:107] Waiting for socket to connect
0809 22:53:12.835417 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 22:53:12.835562 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:53:12.844474 (+ 8912us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:53:12.844480 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:53:12.846996 (+ 2516us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:53:12.847673 (+ 677us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:53:12.847688 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:53:12.851339 (+ 3651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:53:12.851344 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:53:12.856098 (+ 4754us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:53:12.856113 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:53:12.863729 (+ 7616us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:53:12.863735 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:53:12.863958 (+ 223us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:12.864786 (+ 828us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:53:12.864808 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:53:12.868497 (+ 3689us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:53:12.872709 (+ 4212us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:53:12.872714 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:53:12.872719 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:53:12.873284 (+ 565us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:53:12.873751 (+ 467us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:53:12.873757 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:53:12.873761 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:53:12.874074 (+ 313us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:53:12.874586 (+ 512us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:53:12.874591 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:53:12.874821 (+ 230us) client_negotiation.cc:769] Sending connection context
0809 22:53:12.874961 (+ 140us) client_negotiation.cc:240] Negotiation successful
0809 22:53:12.875314 (+ 353us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17945,"thread_start_us":23256,"threads_started":1}
I20260809 22:53:12.877166 27360 heartbeater.cc:347] Connected to a master server at 127.15.113.189:41115
I20260809 22:53:12.877398 27360 heartbeater.cc:464] Registering TS with master...
I20260809 22:53:12.878260 27360 heartbeater.cc:511] Master 127.15.113.189:41115 requested a full tablet report, sending...
I20260809 22:53:12.880558 26748 ts_manager.cc:194] Registered new tserver with Master: b8e298532adf4449955b1b5dc49552c2 (127.15.113.129:38043)
I20260809 22:53:12.950007 15814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskItYPJR/build/asan/bin/kudu with pid 26101
I20260809 22:53:12.975304 15814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskItYPJR/build/asan/bin/kudu
/tmp/dist-test-taskItYPJR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315935617919-15814-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315935617919-15814-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315935617919-15814-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315935617919-15814-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.113.130:37171
--local_ip_for_outbound_sockets=127.15.113.130
--tserver_master_addrs=127.15.113.190:42225
--webserver_port=34741
--webserver_interface=127.15.113.130
--keytab_file=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315935617919-15814-0/krb5kdc/oryx_127.15.113.130.keytab
--principal=oryx/127.15.113.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.113.148:37185
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.15.113.190:42225,127.15.113.189:41115
--tserver_master_addrs=127.15.113.190:42225,127.15.113.189:41115,127.15.113.188:43669 with env {KRB5CCNAME=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315935617919-15814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315935617919-15814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315935617919-15814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 22:53:13.255236 27375 sys_catalog.cc:455] T 00000000000000000000000000000000 P 460ec5412d65461a811d766224ae4ab6 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "460ec5412d65461a811d766224ae4ab6" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "460ec5412d65461a811d766224ae4ab6" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 42225 } } peers { permanent_uuid: "87d4ebb338564421816a43cffeb42715" member_type: VOTER last_known_addr { host: "127.15.113.189" port: 41115 } attrs { promote: false } } peers { permanent_uuid: "70317072228d4a47877776b7fb4e21d1" member_type: NON_VOTER last_known_addr { host: "127.15.113.188" port: 43669 } attrs { promote: true } } }
I20260809 22:53:13.255597 27375 sys_catalog.cc:458] T 00000000000000000000000000000000 P 460ec5412d65461a811d766224ae4ab6 [sys.catalog]: This master's current role is: LEADER
I20260809 22:53:13.257112 27379 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 460ec5412d65461a811d766224ae4ab6: attempting to promote NON_VOTER 70317072228d4a47877776b7fb4e21d1 to VOTER
I20260809 22:53:13.257966 27379 consensus_queue.cc:237] T 00000000000000000000000000000000 P 460ec5412d65461a811d766224ae4ab6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "460ec5412d65461a811d766224ae4ab6" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 42225 } } peers { permanent_uuid: "87d4ebb338564421816a43cffeb42715" member_type: VOTER last_known_addr { host: "127.15.113.189" port: 41115 } attrs { promote: false } } peers { permanent_uuid: "70317072228d4a47877776b7fb4e21d1" member_type: VOTER last_known_addr { host: "127.15.113.188" port: 43669 } attrs { promote: false } }
I20260809 22:53:13.260314 27070 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 70317072228d4a47877776b7fb4e21d1 [term 1 LEARNER]: Refusing update from remote peer 460ec5412d65461a811d766224ae4ab6: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20260809 22:53:13.260710 26762 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 87d4ebb338564421816a43cffeb42715 [term 1 FOLLOWER]: Refusing update from remote peer 460ec5412d65461a811d766224ae4ab6: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20260809 22:53:13.260969 27377 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 460ec5412d65461a811d766224ae4ab6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "70317072228d4a47877776b7fb4e21d1" member_type: VOTER last_known_addr { host: "127.15.113.188" port: 43669 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260809 22:53:13.261497 27379 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 460ec5412d65461a811d766224ae4ab6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "87d4ebb338564421816a43cffeb42715" member_type: VOTER last_known_addr { host: "127.15.113.189" port: 41115 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260809 22:53:13.264458 27379 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 460ec5412d65461a811d766224ae4ab6 [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 70317072228d4a47877776b7fb4e21d1 (127.15.113.188) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "460ec5412d65461a811d766224ae4ab6" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 42225 } } peers { permanent_uuid: "87d4ebb338564421816a43cffeb42715" member_type: VOTER last_known_addr { host: "127.15.113.189" port: 41115 } attrs { promote: false } } peers { permanent_uuid: "70317072228d4a47877776b7fb4e21d1" member_type: VOTER last_known_addr { host: "127.15.113.188" port: 43669 } attrs { promote: false } } }
I20260809 22:53:13.265206 26762 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 87d4ebb338564421816a43cffeb42715 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 70317072228d4a47877776b7fb4e21d1 (127.15.113.188) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "460ec5412d65461a811d766224ae4ab6" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 42225 } } peers { permanent_uuid: "87d4ebb338564421816a43cffeb42715" member_type: VOTER last_known_addr { host: "127.15.113.189" port: 41115 } attrs { promote: false } } peers { permanent_uuid: "70317072228d4a47877776b7fb4e21d1" member_type: VOTER last_known_addr { host: "127.15.113.188" port: 43669 } attrs { promote: false } } }
I20260809 22:53:13.268622 27377 sys_catalog.cc:455] T 00000000000000000000000000000000 P 460ec5412d65461a811d766224ae4ab6 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "460ec5412d65461a811d766224ae4ab6" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "460ec5412d65461a811d766224ae4ab6" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 42225 } } peers { permanent_uuid: "87d4ebb338564421816a43cffeb42715" member_type: VOTER last_known_addr { host: "127.15.113.189" port: 41115 } attrs { promote: false } } peers { permanent_uuid: "70317072228d4a47877776b7fb4e21d1" member_type: VOTER last_known_addr { host: "127.15.113.188" port: 43669 } attrs { promote: false } } }
I20260809 22:53:13.268718 27375 sys_catalog.cc:455] T 00000000000000000000000000000000 P 460ec5412d65461a811d766224ae4ab6 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "460ec5412d65461a811d766224ae4ab6" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "460ec5412d65461a811d766224ae4ab6" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 42225 } } peers { permanent_uuid: "87d4ebb338564421816a43cffeb42715" member_type: VOTER last_known_addr { host: "127.15.113.189" port: 41115 } attrs { promote: false } } peers { permanent_uuid: "70317072228d4a47877776b7fb4e21d1" member_type: VOTER last_known_addr { host: "127.15.113.188" port: 43669 } attrs { promote: false } } }
I20260809 22:53:13.268975 27377 sys_catalog.cc:458] T 00000000000000000000000000000000 P 460ec5412d65461a811d766224ae4ab6 [sys.catalog]: This master's current role is: LEADER
I20260809 22:53:13.269006 27375 sys_catalog.cc:458] T 00000000000000000000000000000000 P 460ec5412d65461a811d766224ae4ab6 [sys.catalog]: This master's current role is: LEADER
I20260809 22:53:13.269193 27377 sys_catalog.cc:455] T 00000000000000000000000000000000 P 460ec5412d65461a811d766224ae4ab6 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "460ec5412d65461a811d766224ae4ab6" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "460ec5412d65461a811d766224ae4ab6" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 42225 } } peers { permanent_uuid: "87d4ebb338564421816a43cffeb42715" member_type: VOTER last_known_addr { host: "127.15.113.189" port: 41115 } attrs { promote: false } } peers { permanent_uuid: "70317072228d4a47877776b7fb4e21d1" member_type: VOTER last_known_addr { host: "127.15.113.188" port: 43669 } attrs { promote: false } } }
I20260809 22:53:13.269404 27377 sys_catalog.cc:458] T 00000000000000000000000000000000 P 460ec5412d65461a811d766224ae4ab6 [sys.catalog]: This master's current role is: LEADER
I20260809 22:53:13.267827 27070 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 70317072228d4a47877776b7fb4e21d1 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 70317072228d4a47877776b7fb4e21d1 (127.15.113.188) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "460ec5412d65461a811d766224ae4ab6" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 42225 } } peers { permanent_uuid: "87d4ebb338564421816a43cffeb42715" member_type: VOTER last_known_addr { host: "127.15.113.189" port: 41115 } attrs { promote: false } } peers { permanent_uuid: "70317072228d4a47877776b7fb4e21d1" member_type: VOTER last_known_addr { host: "127.15.113.188" port: 43669 } attrs { promote: false } } }
I20260809 22:53:13.277309 27391 sys_catalog.cc:455] T 00000000000000000000000000000000 P 87d4ebb338564421816a43cffeb42715 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "460ec5412d65461a811d766224ae4ab6" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "460ec5412d65461a811d766224ae4ab6" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 42225 } } peers { permanent_uuid: "87d4ebb338564421816a43cffeb42715" member_type: VOTER last_known_addr { host: "127.15.113.189" port: 41115 } attrs { promote: false } } peers { permanent_uuid: "70317072228d4a47877776b7fb4e21d1" member_type: VOTER last_known_addr { host: "127.15.113.188" port: 43669 } attrs { promote: false } } }
I20260809 22:53:13.277729 27391 sys_catalog.cc:458] T 00000000000000000000000000000000 P 87d4ebb338564421816a43cffeb42715 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 22:53:13.281395 27392 sys_catalog.cc:455] T 00000000000000000000000000000000 P 70317072228d4a47877776b7fb4e21d1 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "460ec5412d65461a811d766224ae4ab6" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "460ec5412d65461a811d766224ae4ab6" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 42225 } } peers { permanent_uuid: "87d4ebb338564421816a43cffeb42715" member_type: VOTER last_known_addr { host: "127.15.113.189" port: 41115 } attrs { promote: false } } peers { permanent_uuid: "70317072228d4a47877776b7fb4e21d1" member_type: VOTER last_known_addr { host: "127.15.113.188" port: 43669 } attrs { promote: false } } }
I20260809 22:53:13.281769 27392 sys_catalog.cc:458] T 00000000000000000000000000000000 P 70317072228d4a47877776b7fb4e21d1 [sys.catalog]: This master's current role is: FOLLOWER
W20260809 22:53:13.313992 27374 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:53:13.314244 27374 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:53:13.314379 27374 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:53:13.321355 27374 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:53:13.321452 27374 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:53:13.321596 27374 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.113.130
I20260809 22:53:13.331101 27374 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.113.148:37185
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315935617919-15814-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315935617919-15814-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315935617919-15814-0/krb5kdc/oryx_127.15.113.130.keytab
--principal=oryx/127.15.113.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.113.130:37171
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315935617919-15814-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.113.130
--webserver_port=34741
--tserver_master_addrs=127.15.113.190:42225,127.15.113.189:41115,127.15.113.188:43669
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.113.130
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315935617919-15814-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:53:13.332288 27374 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:53:13.333679 27374 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:53:13.347366 27401 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:53:13.347473 27400 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:53:13.347494 27404 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:53:13.348565 27374 server_base.cc:1034] running on GCE node
I20260809 22:53:13.349177 27374 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:53:13.350072 27374 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:53:13.351264 27374 hybrid_clock.cc:648] HybridClock initialized: now 1786315993351219 us; error 38 us; skew 500 ppm
I20260809 22:53:13.351558 27374 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:53:13 dist-test-slave-ncp9 krb5kdc[25693](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315993, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.113.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:53:13.354578 27374 init.cc:377] Logged in from keytab as oryx/127.15.113.130@KRBTEST.COM (short username oryx)
I20260809 22:53:13.356194 27374 webserver.cc:460] Webserver started at http://127.15.113.130:34741/ using document root <none> and password file <none>
I20260809 22:53:13.356812 27374 fs_manager.cc:359] Metadata directory not provided
I20260809 22:53:13.356971 27374 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:53:13.362293 27374 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.000s sys 0.004s
I20260809 22:53:13.364853 27415 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:53:13.366190 27374 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260809 22:53:13.366326 27374 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315935617919-15814-0/minicluster-data/ts-1/data,/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315935617919-15814-0/minicluster-data/ts-1/wal
uuid: "200f1dd145df4993b427e20748be2114"
format_stamp: "Formatted at 2026-08-09 22:53:09 on dist-test-slave-ncp9"
server_key: "ed2210da7cb7d050dcc4392744efa732"
server_key_iv: "835280f3abcd0799fa0bf6da6eb0daa3"
server_key_version: "encryptionkey@0"
I20260809 22:53:13.367280 27374 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315935617919-15814-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315935617919-15814-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315935617919-15814-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:53:13.386353 27374 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:53:13.390050 27374 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:53:13.390291 27374 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:53:13.391130 27374 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:53:13.392380 27374 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:53:13.392463 27374 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:53:13.392544 27374 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:53:13.392597 27374 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:53:13.422333 27374 rpc_server.cc:307] RPC server started. Bound to: 127.15.113.130:37171
I20260809 22:53:13.422398 27589 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.113.130:37171 every 8 connection(s)
I20260809 22:53:13.423730 27374 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315935617919-15814-0/minicluster-data/ts-1/data/info.pb
I20260809 22:53:13.429385 15814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskItYPJR/build/asan/bin/kudu as pid 27374
I20260809 22:53:13.429706 15814 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskItYPJR/build/asan/bin/kudu pid 26313
Aug 09 22:53:13 dist-test-slave-ncp9 krb5kdc[25693](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315993, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.113.130@KRBTEST.COM for oryx/127.15.113.190@KRBTEST.COM
Aug 09 22:53:13 dist-test-slave-ncp9 krb5kdc[25693](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315993, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.113.130@KRBTEST.COM for oryx/127.15.113.188@KRBTEST.COM
Aug 09 22:53:13 dist-test-slave-ncp9 krb5kdc[25693](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315993, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.113.130@KRBTEST.COM for oryx/127.15.113.189@KRBTEST.COM
Aug 09 22:53:13 dist-test-slave-ncp9 krb5kdc[25693](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315988, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.15.113.131@KRBTEST.COM
I20260809 22:53:13.477598 27601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:13.426556 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.113.130:45615 (local address 127.15.113.189:41115)
0809 22:53:13.449694 (+ 23138us) server_negotiation.cc:206] Beginning negotiation
0809 22:53:13.449702 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 22:53:13.450590 (+ 888us) server_negotiation.cc:407] Connection header received
0809 22:53:13.450671 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:53:13.450676 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:53:13.450779 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:53:13.450896 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:53:13.455397 (+ 4501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:13.457197 (+ 1800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:13.459945 (+ 2748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:13.461077 (+ 1132us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:13.461206 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:13.468339 (+ 7133us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:53:13.468348 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:53:13.468352 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:53:13.468399 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:53:13.473050 (+ 4651us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:53:13.474528 (+ 1478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:53:13.474534 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:53:13.474538 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:53:13.474825 (+ 287us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:53:13.475345 (+ 520us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:53:13.475350 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:53:13.475353 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:53:13.475740 (+ 387us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:53:13.475889 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0809 22:53:13.476771 (+ 882us) server_negotiation.cc:299] Negotiation successful
0809 22:53:13.477047 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22994,"thread_start_us":22856,"threads_started":1}
I20260809 22:53:13.478796 27612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:13.450720 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.113.130:47769 (local address 127.15.113.188:43669)
0809 22:53:13.451765 (+ 1045us) server_negotiation.cc:206] Beginning negotiation
0809 22:53:13.451771 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:53:13.453333 (+ 1562us) server_negotiation.cc:407] Connection header received
0809 22:53:13.453625 (+ 292us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:53:13.453630 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:53:13.453709 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:53:13.453791 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:53:13.455448 (+ 1657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:13.456562 (+ 1114us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:13.459391 (+ 2829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:13.460511 (+ 1120us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:13.460609 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:13.468354 (+ 7745us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:53:13.468363 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:53:13.468368 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:53:13.468411 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:53:13.473414 (+ 5003us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:53:13.475330 (+ 1916us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:53:13.475336 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:53:13.475339 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:53:13.475561 (+ 222us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:53:13.476734 (+ 1173us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:53:13.476740 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:53:13.476743 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:53:13.477239 (+ 496us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:53:13.477368 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0809 22:53:13.478057 (+ 689us) server_negotiation.cc:299] Negotiation successful
0809 22:53:13.478329 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":921,"thread_start_us":801,"threads_started":1}
I20260809 22:53:13.479524 27602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:13.427139 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.189:41115 (local address 127.15.113.130:45615)
0809 22:53:13.450130 (+ 22991us) negotiation.cc:107] Waiting for socket to connect
0809 22:53:13.450152 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0809 22:53:13.450292 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:53:13.451157 (+ 865us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:53:13.451164 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:53:13.454492 (+ 3328us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:53:13.455184 (+ 692us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:53:13.455205 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:53:13.457428 (+ 2223us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:53:13.457434 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:53:13.459694 (+ 2260us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:53:13.459712 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:53:13.462525 (+ 2813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:53:13.462538 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:53:13.462747 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:13.463635 (+ 888us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:53:13.463660 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:53:13.468055 (+ 4395us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:53:13.473229 (+ 5174us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:53:13.473239 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:53:13.473257 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:53:13.474375 (+ 1118us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:53:13.474964 (+ 589us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:53:13.474970 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:53:13.474975 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:53:13.475188 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:53:13.475889 (+ 701us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:53:13.475898 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:53:13.476311 (+ 413us) client_negotiation.cc:769] Sending connection context
0809 22:53:13.476645 (+ 334us) client_negotiation.cc:240] Negotiation successful
0809 22:53:13.477198 (+ 553us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":22842,"mutex_wait_us":32,"spinlock_wait_cycles":39424,"thread_start_us":17724,"threads_started":1}
I20260809 22:53:13.480540 27599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:13.426473 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.113.130:50221 (local address 127.15.113.190:42225)
0809 22:53:13.448642 (+ 22169us) server_negotiation.cc:206] Beginning negotiation
0809 22:53:13.448648 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:53:13.448737 (+ 89us) server_negotiation.cc:407] Connection header received
0809 22:53:13.448835 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:53:13.448842 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:53:13.448991 (+ 149us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:53:13.449122 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:53:13.454810 (+ 5688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:13.456020 (+ 1210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:13.458960 (+ 2940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:13.459985 (+ 1025us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:13.460109 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:13.467187 (+ 7078us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:53:13.467195 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:53:13.467201 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:53:13.467286 (+ 85us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:53:13.474363 (+ 7077us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:53:13.475306 (+ 943us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:53:13.475312 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:53:13.475316 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:53:13.475568 (+ 252us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:53:13.477958 (+ 2390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:53:13.477964 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:53:13.477967 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:53:13.478470 (+ 503us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:53:13.478633 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0809 22:53:13.479696 (+ 1063us) server_negotiation.cc:299] Negotiation successful
0809 22:53:13.479927 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22027,"thread_start_us":23976,"threads_started":1}
I20260809 22:53:13.480657 27608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:13.427128 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:42225 (local address 127.15.113.130:50221)
0809 22:53:13.441546 (+ 14418us) negotiation.cc:107] Waiting for socket to connect
0809 22:53:13.441599 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0809 22:53:13.444589 (+ 2990us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:53:13.451012 (+ 6423us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:53:13.451025 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:53:13.452125 (+ 1100us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:53:13.453518 (+ 1393us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:53:13.453548 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:53:13.456258 (+ 2710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:53:13.456265 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:53:13.458769 (+ 2504us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:53:13.458786 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:53:13.460643 (+ 1857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:53:13.460656 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:53:13.461118 (+ 462us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:13.462224 (+ 1106us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:53:13.462318 (+ 94us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:53:13.466857 (+ 4539us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:53:13.474550 (+ 7693us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:53:13.474555 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:53:13.474559 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:53:13.475158 (+ 599us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:53:13.476227 (+ 1069us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:53:13.476232 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:53:13.476236 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:53:13.476561 (+ 325us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:53:13.479029 (+ 2468us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:53:13.479034 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:53:13.479284 (+ 250us) client_negotiation.cc:769] Sending connection context
0809 22:53:13.479430 (+ 146us) client_negotiation.cc:240] Negotiation successful
0809 22:53:13.479768 (+ 338us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14084,"mutex_wait_us":31,"spinlock_wait_cycles":3712,"thread_start_us":22799,"threads_started":1}
I20260809 22:53:13.482141 27616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:13.451991 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.188:43669 (local address 127.15.113.130:47769)
0809 22:53:13.453240 (+ 1249us) negotiation.cc:107] Waiting for socket to connect
0809 22:53:13.453261 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0809 22:53:13.453467 (+ 206us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:53:13.454042 (+ 575us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:53:13.454047 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:53:13.454669 (+ 622us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:53:13.455261 (+ 592us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:53:13.455277 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:53:13.456746 (+ 1469us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:53:13.456751 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:53:13.459235 (+ 2484us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:53:13.459251 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:53:13.461078 (+ 1827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:53:13.461087 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:53:13.461420 (+ 333us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:13.463101 (+ 1681us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:53:13.463122 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:53:13.468039 (+ 4917us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:53:13.473581 (+ 5542us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:53:13.473586 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:53:13.473590 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:53:13.474433 (+ 843us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:53:13.475741 (+ 1308us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:53:13.475746 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:53:13.475749 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:53:13.476107 (+ 358us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:53:13.477456 (+ 1349us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:53:13.477461 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:53:13.477667 (+ 206us) client_negotiation.cc:769] Sending connection context
0809 22:53:13.480880 (+ 3213us) client_negotiation.cc:240] Negotiation successful
0809 22:53:13.481159 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1066,"mutex_wait_us":27,"thread_start_us":2567,"threads_started":1}
I20260809 22:53:13.482369 27611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:13.430596 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58096 (local address 127.15.113.131:32941)
0809 22:53:13.452812 (+ 22216us) server_negotiation.cc:206] Beginning negotiation
0809 22:53:13.452820 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 22:53:13.452852 (+ 32us) server_negotiation.cc:407] Connection header received
0809 22:53:13.452941 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:53:13.452950 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:53:13.453199 (+ 249us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:53:13.453360 (+ 161us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:53:13.456443 (+ 3083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:13.457962 (+ 1519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:13.460260 (+ 2298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:13.461455 (+ 1195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:13.461663 (+ 208us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:13.471598 (+ 9935us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:53:13.471613 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:53:13.471633 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:53:13.471674 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:53:13.476846 (+ 5172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:53:13.477802 (+ 956us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:53:13.477810 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:53:13.477814 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:53:13.478099 (+ 285us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:53:13.478662 (+ 563us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:53:13.478667 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:53:13.478670 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:53:13.479169 (+ 499us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:53:13.479491 (+ 322us) server_negotiation.cc:1035] Waiting for connection context
0809 22:53:13.481393 (+ 1902us) server_negotiation.cc:299] Negotiation successful
0809 22:53:13.481658 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22027,"thread_start_us":21857,"threads_started":1}
I20260809 22:53:13.482419 27593 heartbeater.cc:347] Connected to a master server at 127.15.113.190:42225
I20260809 22:53:13.482331 27594 heartbeater.cc:347] Connected to a master server at 127.15.113.189:41115
I20260809 22:53:13.482832 27594 heartbeater.cc:464] Registering TS with master...
I20260809 22:53:13.482829 27593 heartbeater.cc:464] Registering TS with master...
I20260809 22:53:13.483966 27592 heartbeater.cc:347] Connected to a master server at 127.15.113.188:43669
I20260809 22:53:13.484090 26515 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:53:13.484212 27592 heartbeater.cc:464] Registering TS with master...
I20260809 22:53:13.484447 27593 heartbeater.cc:511] Master 127.15.113.190:42225 requested a full tablet report, sending...
I20260809 22:53:13.486665 25770 ts_manager.cc:194] Re-registered known tserver with Master: 200f1dd145df4993b427e20748be2114 (127.15.113.130:37171)
I20260809 22:53:13.487917 25770 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.15.113.130@KRBTEST.COM'} at 127.15.113.130:50221
I20260809 22:53:13.489264 27592 heartbeater.cc:511] Master 127.15.113.188:43669 requested a full tablet report, sending...
I20260809 22:53:13.490255 27594 heartbeater.cc:511] Master 127.15.113.189:41115 requested a full tablet report, sending...
I20260809 22:53:13.491393 27057 ts_manager.cc:194] Registered new tserver with Master: 200f1dd145df4993b427e20748be2114 (127.15.113.130:37171)
I20260809 22:53:13.491688 26748 ts_manager.cc:194] Registered new tserver with Master: 200f1dd145df4993b427e20748be2114 (127.15.113.130:37171)
I20260809 22:53:13.575788 15814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskItYPJR/build/asan/bin/kudu with pid 26313
I20260809 22:53:13.595749 15814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskItYPJR/build/asan/bin/kudu
/tmp/dist-test-taskItYPJR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315935617919-15814-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315935617919-15814-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315935617919-15814-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315935617919-15814-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.113.131:32941
--local_ip_for_outbound_sockets=127.15.113.131
--tserver_master_addrs=127.15.113.190:42225
--webserver_port=34131
--webserver_interface=127.15.113.131
--keytab_file=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315935617919-15814-0/krb5kdc/oryx_127.15.113.131.keytab
--principal=oryx/127.15.113.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.113.148:37185
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.15.113.190:42225,127.15.113.189:41115
--tserver_master_addrs=127.15.113.190:42225,127.15.113.189:41115,127.15.113.188:43669 with env {KRB5CCNAME=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315935617919-15814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315935617919-15814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315935617919-15814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 22:53:13.867408 27353 heartbeater.cc:503] Master 127.15.113.190:42225 was elected leader, sending a full tablet report...
W20260809 22:53:13.934958 27618 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:53:13.935236 27618 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:53:13.935343 27618 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:53:13.942504 27618 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:53:13.942613 27618 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:53:13.942737 27618 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.113.131
I20260809 22:53:13.952155 27618 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.113.148:37185
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315935617919-15814-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315935617919-15814-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315935617919-15814-0/krb5kdc/oryx_127.15.113.131.keytab
--principal=oryx/127.15.113.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.113.131:32941
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315935617919-15814-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.113.131
--webserver_port=34131
--tserver_master_addrs=127.15.113.190:42225,127.15.113.189:41115,127.15.113.188:43669
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.113.131
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315935617919-15814-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:53:13.953418 27618 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:53:13.954702 27618 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:53:13.969696 27626 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:53:13.969717 27627 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:53:13.969640 27629 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:53:13.970561 27618 server_base.cc:1034] running on GCE node
I20260809 22:53:13.971145 27618 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:53:13.972052 27618 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:53:13.973237 27618 hybrid_clock.cc:648] HybridClock initialized: now 1786315993973186 us; error 36 us; skew 500 ppm
I20260809 22:53:13.973541 27618 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:53:13 dist-test-slave-ncp9 krb5kdc[25693](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315993, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.113.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:53:13.976531 27618 init.cc:377] Logged in from keytab as oryx/127.15.113.131@KRBTEST.COM (short username oryx)
I20260809 22:53:13.977936 27618 webserver.cc:460] Webserver started at http://127.15.113.131:34131/ using document root <none> and password file <none>
I20260809 22:53:13.978608 27618 fs_manager.cc:359] Metadata directory not provided
I20260809 22:53:13.978739 27618 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:53:13.984091 27618 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.002s
I20260809 22:53:13.986656 27637 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:53:13.987876 27618 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 22:53:13.988047 27618 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315935617919-15814-0/minicluster-data/ts-2/data,/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315935617919-15814-0/minicluster-data/ts-2/wal
uuid: "a0eed0b7d94e4c1f905abd17c6d0fd80"
format_stamp: "Formatted at 2026-08-09 22:53:10 on dist-test-slave-ncp9"
server_key: "8b80b276f3fbc09d710c9fd7a50a134c"
server_key_iv: "25092b0b69bef32fe7df021917b0f828"
server_key_version: "encryptionkey@0"
I20260809 22:53:13.989110 27618 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315935617919-15814-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315935617919-15814-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315935617919-15814-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:53:14.016517 27618 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:53:14.020223 27618 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:53:14.020498 27618 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:53:14.021442 27618 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:53:14.022980 27618 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:53:14.023083 27618 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:53:14.023200 27618 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:53:14.023270 27618 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:53:14.054890 27618 rpc_server.cc:307] RPC server started. Bound to: 127.15.113.131:32941
I20260809 22:53:14.054944 27831 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.113.131:32941 every 8 connection(s)
I20260809 22:53:14.056365 27618 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315935617919-15814-0/minicluster-data/ts-2/data/info.pb
I20260809 22:53:14.059561 15814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskItYPJR/build/asan/bin/kudu as pid 27618
Aug 09 22:53:14 dist-test-slave-ncp9 krb5kdc[25693](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315993, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.113.131@KRBTEST.COM for oryx/127.15.113.190@KRBTEST.COM
Aug 09 22:53:14 dist-test-slave-ncp9 krb5kdc[25693](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315993, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.113.131@KRBTEST.COM for oryx/127.15.113.188@KRBTEST.COM
Aug 09 22:53:14 dist-test-slave-ncp9 krb5kdc[25693](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315993, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.113.131@KRBTEST.COM for oryx/127.15.113.189@KRBTEST.COM
I20260809 22:53:14.101554 27837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:14.059787 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.188:43669 (local address 127.15.113.131:45743)
0809 22:53:14.060702 (+ 915us) negotiation.cc:107] Waiting for socket to connect
0809 22:53:14.060756 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0809 22:53:14.071894 (+ 11138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:53:14.074479 (+ 2585us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:53:14.074497 (+ 18us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:53:14.075336 (+ 839us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:53:14.076292 (+ 956us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:53:14.076319 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:53:14.078041 (+ 1722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:53:14.078046 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:53:14.080663 (+ 2617us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:53:14.080675 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:53:14.082075 (+ 1400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:53:14.082081 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:53:14.082450 (+ 369us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:14.083499 (+ 1049us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:53:14.083537 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:53:14.090106 (+ 6569us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:53:14.095931 (+ 5825us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:53:14.095941 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:53:14.095957 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:53:14.096577 (+ 620us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:53:14.097289 (+ 712us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:53:14.097295 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:53:14.097299 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:53:14.097546 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:53:14.098800 (+ 1254us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:53:14.098809 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:53:14.099285 (+ 476us) client_negotiation.cc:769] Sending connection context
0809 22:53:14.099592 (+ 307us) client_negotiation.cc:240] Negotiation successful
0809 22:53:14.100131 (+ 539us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":584,"mutex_wait_us":54,"spinlock_wait_cycles":9088,"thread_start_us":555,"threads_started":1}
I20260809 22:53:14.102502 27846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:14.061669 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.113.131:48585 (local address 127.15.113.190:42225)
0809 22:53:14.073144 (+ 11475us) server_negotiation.cc:206] Beginning negotiation
0809 22:53:14.073149 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:53:14.074812 (+ 1663us) server_negotiation.cc:407] Connection header received
0809 22:53:14.075111 (+ 299us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:53:14.075117 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:53:14.075200 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:53:14.075288 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:53:14.077030 (+ 1742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:14.078107 (+ 1077us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:14.080520 (+ 2413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:14.081165 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:14.081293 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:14.087836 (+ 6543us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:53:14.087843 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:53:14.087848 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:53:14.087893 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:53:14.092829 (+ 4936us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:53:14.099388 (+ 6559us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:53:14.099394 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:53:14.099398 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:53:14.099591 (+ 193us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:53:14.100195 (+ 604us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:53:14.100201 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:53:14.100205 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:53:14.100689 (+ 484us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:53:14.100831 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0809 22:53:14.101601 (+ 770us) server_negotiation.cc:299] Negotiation successful
0809 22:53:14.101882 (+ 281us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11373,"thread_start_us":11236,"threads_started":1}
I20260809 22:53:14.102870 27841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:14.061069 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.113.131:45743 (local address 127.15.113.188:43669)
0809 22:53:14.073805 (+ 12736us) server_negotiation.cc:206] Beginning negotiation
0809 22:53:14.073812 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:53:14.073834 (+ 22us) server_negotiation.cc:407] Connection header received
0809 22:53:14.073890 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:53:14.073894 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:53:14.073980 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:53:14.074092 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:53:14.076548 (+ 2456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:14.077805 (+ 1257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:14.080832 (+ 3027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:14.081503 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:14.081610 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:14.090377 (+ 8767us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:53:14.090385 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:53:14.090390 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:53:14.090435 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:53:14.095759 (+ 5324us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:53:14.096984 (+ 1225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:53:14.096990 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:53:14.096994 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:53:14.097146 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:53:14.097688 (+ 542us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:53:14.097693 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:53:14.097697 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:53:14.098039 (+ 342us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:53:14.098173 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0809 22:53:14.102054 (+ 3881us) server_negotiation.cc:299] Negotiation successful
0809 22:53:14.102293 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12602,"thread_start_us":12484,"threads_started":1}
I20260809 22:53:14.103564 27839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:14.061508 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:42225 (local address 127.15.113.131:48585)
0809 22:53:14.074703 (+ 13195us) negotiation.cc:107] Waiting for socket to connect
0809 22:53:14.074722 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 22:53:14.074929 (+ 207us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:53:14.075536 (+ 607us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:53:14.075541 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:53:14.076153 (+ 612us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:53:14.076792 (+ 639us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:53:14.076807 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:53:14.078302 (+ 1495us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:53:14.078308 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:53:14.080342 (+ 2034us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:53:14.080361 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:53:14.081777 (+ 1416us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:53:14.081792 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:53:14.082591 (+ 799us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:14.083978 (+ 1387us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:53:14.084002 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:53:14.087248 (+ 3246us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:53:14.098482 (+ 11234us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:53:14.098488 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:53:14.098492 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:53:14.099260 (+ 768us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:53:14.099769 (+ 509us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:53:14.099779 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:53:14.099784 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:53:14.100026 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:53:14.100961 (+ 935us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:53:14.100966 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:53:14.101196 (+ 230us) client_negotiation.cc:769] Sending connection context
0809 22:53:14.102757 (+ 1561us) client_negotiation.cc:240] Negotiation successful
0809 22:53:14.103027 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13074,"mutex_wait_us":817,"thread_start_us":11085,"threads_started":1}
I20260809 22:53:14.104543 27832 heartbeater.cc:347] Connected to a master server at 127.15.113.188:43669
I20260809 22:53:14.104964 27832 heartbeater.cc:464] Registering TS with master...
I20260809 22:53:14.106328 27832 heartbeater.cc:511] Master 127.15.113.188:43669 requested a full tablet report, sending...
I20260809 22:53:14.106559 27833 heartbeater.cc:347] Connected to a master server at 127.15.113.190:42225
I20260809 22:53:14.106751 27833 heartbeater.cc:464] Registering TS with master...
I20260809 22:53:14.107399 27833 heartbeater.cc:511] Master 127.15.113.190:42225 requested a full tablet report, sending...
I20260809 22:53:14.109565 25770 ts_manager.cc:194] Re-registered known tserver with Master: a0eed0b7d94e4c1f905abd17c6d0fd80 (127.15.113.131:32941)
I20260809 22:53:14.110970 25770 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.15.113.131@KRBTEST.COM'} at 127.15.113.131:48585
I20260809 22:53:14.113445 27057 ts_manager.cc:194] Registered new tserver with Master: a0eed0b7d94e4c1f905abd17c6d0fd80 (127.15.113.131:32941)
I20260809 22:53:14.122524 27843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:14.059788 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.189:41115 (local address 127.15.113.131:60735)
0809 22:53:14.072353 (+ 12565us) negotiation.cc:107] Waiting for socket to connect
0809 22:53:14.072376 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0809 22:53:14.072502 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:53:14.080387 (+ 7885us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:53:14.080391 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:53:14.080985 (+ 594us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:53:14.081867 (+ 882us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:53:14.081882 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:53:14.092010 (+ 10128us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:53:14.092017 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:53:14.094310 (+ 2293us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:53:14.094327 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:53:14.096074 (+ 1747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:53:14.096085 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:53:14.096401 (+ 316us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:14.097168 (+ 767us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:53:14.097195 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:53:14.102992 (+ 5797us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:53:14.119081 (+ 16089us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:53:14.119087 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:53:14.119090 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:53:14.119576 (+ 486us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:53:14.120064 (+ 488us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:53:14.120069 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:53:14.120073 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:53:14.120304 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:53:14.121066 (+ 762us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:53:14.121071 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:53:14.121253 (+ 182us) client_negotiation.cc:769] Sending connection context
0809 22:53:14.121369 (+ 116us) client_negotiation.cc:240] Negotiation successful
0809 22:53:14.121653 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12400,"mutex_wait_us":78,"spinlock_wait_cycles":73984,"thread_start_us":14454,"threads_started":1}
I20260809 22:53:14.122797 27852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:14.082091 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52230 (local address 127.15.113.190:42225)
0809 22:53:14.084019 (+ 1928us) server_negotiation.cc:206] Beginning negotiation
0809 22:53:14.084025 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:53:14.084045 (+ 20us) server_negotiation.cc:407] Connection header received
0809 22:53:14.084097 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:53:14.084102 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:53:14.084261 (+ 159us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:53:14.084379 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:53:14.090362 (+ 5983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:14.094078 (+ 3716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:14.096049 (+ 1971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:14.097067 (+ 1018us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:14.097199 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:14.113170 (+ 15971us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:53:14.113176 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:53:14.113181 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:53:14.113214 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:53:14.117969 (+ 4755us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:53:14.119347 (+ 1378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:53:14.119352 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:53:14.119356 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:53:14.119576 (+ 220us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:53:14.120197 (+ 621us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:53:14.120202 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:53:14.120205 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:53:14.120642 (+ 437us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:53:14.120860 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0809 22:53:14.121908 (+ 1048us) server_negotiation.cc:299] Negotiation successful
0809 22:53:14.122171 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1755,"thread_start_us":1625,"threads_started":1}
I20260809 22:53:14.124053 27840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:14.060408 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.113.131:60735 (local address 127.15.113.189:41115)
0809 22:53:14.079942 (+ 19534us) server_negotiation.cc:206] Beginning negotiation
0809 22:53:14.079948 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:53:14.079971 (+ 23us) server_negotiation.cc:407] Connection header received
0809 22:53:14.080034 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:53:14.080040 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:53:14.080120 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:53:14.084746 (+ 4626us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:53:14.085059 (+ 313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:14.091742 (+ 6683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:14.094568 (+ 2826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:14.095674 (+ 1106us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:14.096761 (+ 1087us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:14.103288 (+ 6527us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:53:14.103296 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:53:14.103301 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:53:14.103353 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:53:14.118907 (+ 15554us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:53:14.119735 (+ 828us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:53:14.119740 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:53:14.119743 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:53:14.119909 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:53:14.120465 (+ 556us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:53:14.120470 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:53:14.120473 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:53:14.120867 (+ 394us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:53:14.123165 (+ 2298us) server_negotiation.cc:1035] Waiting for connection context
0809 22:53:14.123289 (+ 124us) server_negotiation.cc:299] Negotiation successful
0809 22:53:14.123555 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19392,"thread_start_us":19809,"threads_started":1}
I20260809 22:53:14.129009 27841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:14.083444 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37816 (local address 127.15.113.188:43669)
0809 22:53:14.104048 (+ 20604us) server_negotiation.cc:206] Beginning negotiation
0809 22:53:14.104053 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:53:14.104070 (+ 17us) server_negotiation.cc:407] Connection header received
0809 22:53:14.104113 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:53:14.104117 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:53:14.104192 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:53:14.104279 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:53:14.108049 (+ 3770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:14.109299 (+ 1250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:14.111954 (+ 2655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:14.112788 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:14.112923 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:14.119283 (+ 6360us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:53:14.119289 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:53:14.119294 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:53:14.119328 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:53:14.123455 (+ 4127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:53:14.124528 (+ 1073us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:53:14.124535 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:53:14.124539 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:53:14.124777 (+ 238us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:53:14.125366 (+ 589us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:53:14.125372 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:53:14.125375 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:53:14.125798 (+ 423us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:53:14.126276 (+ 478us) server_negotiation.cc:1035] Waiting for connection context
0809 22:53:14.128031 (+ 1755us) server_negotiation.cc:299] Negotiation successful
0809 22:53:14.128287 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20501,"thread_start_us":32340,"threads_started":1}
I20260809 22:53:14.129138 27834 heartbeater.cc:347] Connected to a master server at 127.15.113.189:41115
I20260809 22:53:14.129338 27834 heartbeater.cc:464] Registering TS with master...
I20260809 22:53:14.129992 27834 heartbeater.cc:511] Master 127.15.113.189:41115 requested a full tablet report, sending...
I20260809 22:53:14.131459 26748 ts_manager.cc:194] Registered new tserver with Master: a0eed0b7d94e4c1f905abd17c6d0fd80 (127.15.113.131:32941)
I20260809 22:53:14.129927 25770 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52230:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 22:53:14.141589 25770 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 22:53:14.142560 27856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:14.091825 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50736 (local address 127.15.113.189:41115)
0809 22:53:14.114025 (+ 22200us) server_negotiation.cc:206] Beginning negotiation
0809 22:53:14.114030 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:53:14.114044 (+ 14us) server_negotiation.cc:407] Connection header received
0809 22:53:14.114086 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:53:14.114091 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:53:14.114171 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:53:14.114266 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:53:14.120364 (+ 6098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:14.127778 (+ 7414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:14.130211 (+ 2433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:14.131107 (+ 896us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:14.131222 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:14.135112 (+ 3890us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:53:14.135120 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:53:14.135123 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:53:14.135166 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:53:14.139057 (+ 3891us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:53:14.139846 (+ 789us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:53:14.139851 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:53:14.139853 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:53:14.140082 (+ 229us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:53:14.140591 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:53:14.140595 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:53:14.140597 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:53:14.141000 (+ 403us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:53:14.141381 (+ 381us) server_negotiation.cc:1035] Waiting for connection context
0809 22:53:14.141747 (+ 366us) server_negotiation.cc:299] Negotiation successful
0809 22:53:14.142011 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21984,"thread_start_us":21469,"threads_started":1}
I20260809 22:53:14.189615 27878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:14.165338 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39810 (local address 127.15.113.129:38043)
0809 22:53:14.172497 (+ 7159us) server_negotiation.cc:206] Beginning negotiation
0809 22:53:14.172508 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 22:53:14.178645 (+ 6137us) server_negotiation.cc:407] Connection header received
0809 22:53:14.178945 (+ 300us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:53:14.178952 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:53:14.179164 (+ 212us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:53:14.179446 (+ 282us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:53:14.180499 (+ 1053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:14.183074 (+ 2575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:14.186295 (+ 3221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:14.188280 (+ 1985us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:14.188466 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:14.188664 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0809 22:53:14.189018 (+ 354us) server_negotiation.cc:299] Negotiation successful
0809 22:53:14.189236 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6913,"thread_start_us":6684,"threads_started":1}
I20260809 22:53:14.191793 27879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:14.164936 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.129:38043 (local address 127.0.0.1:39810)
0809 22:53:14.178468 (+ 13532us) negotiation.cc:107] Waiting for socket to connect
0809 22:53:14.178524 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0809 22:53:14.178731 (+ 207us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:53:14.179388 (+ 657us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:53:14.179394 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:53:14.179423 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:53:14.180289 (+ 866us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:53:14.180307 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:53:14.183315 (+ 3008us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:53:14.183322 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:53:14.186064 (+ 2742us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:53:14.186082 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:53:14.188555 (+ 2473us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:53:14.188563 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:53:14.188775 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:14.188801 (+ 26us) client_negotiation.cc:769] Sending connection context
0809 22:53:14.188884 (+ 83us) client_negotiation.cc:240] Negotiation successful
0809 22:53:14.189038 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13348,"thread_start_us":14519,"threads_started":1}
I20260809 22:53:14.194312 27877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:14.165508 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.131:32941 (local address 127.0.0.1:58110)
0809 22:53:14.180083 (+ 14575us) negotiation.cc:107] Waiting for socket to connect
0809 22:53:14.180101 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 22:53:14.180239 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:53:14.181563 (+ 1324us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:53:14.181569 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:53:14.181601 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:53:14.182824 (+ 1223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:53:14.182844 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:53:14.186615 (+ 3771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:53:14.186620 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:53:14.190169 (+ 3549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:53:14.190187 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:53:14.193118 (+ 2931us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:53:14.193127 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:53:14.193417 (+ 290us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:14.193451 (+ 34us) client_negotiation.cc:769] Sending connection context
0809 22:53:14.193550 (+ 99us) client_negotiation.cc:240] Negotiation successful
0809 22:53:14.193745 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14377,"thread_start_us":12491,"threads_started":1}
I20260809 22:53:14.194384 27881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:14.166414 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58110 (local address 127.15.113.131:32941)
0809 22:53:14.177613 (+ 11199us) server_negotiation.cc:206] Beginning negotiation
0809 22:53:14.177626 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 22:53:14.180390 (+ 2764us) server_negotiation.cc:407] Connection header received
0809 22:53:14.180531 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:53:14.180542 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:53:14.180814 (+ 272us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:53:14.181072 (+ 258us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:53:14.183207 (+ 2135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:14.186375 (+ 3168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:14.190357 (+ 3982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:14.192775 (+ 2418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:14.193044 (+ 269us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:14.193230 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0809 22:53:14.193663 (+ 433us) server_negotiation.cc:299] Negotiation successful
0809 22:53:14.193880 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10896,"thread_start_us":10428,"threads_started":1}
I20260809 22:53:14.196513 27236 tablet_service.cc:1467] Processing CreateTablet for tablet c525c10d8f0c42cd82c794ba5cff8a6f (DEFAULT_TABLE table=test-table [id=9c6eaa4c8369408c8cce7d3f9908110f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:53:14.198485 27236 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c525c10d8f0c42cd82c794ba5cff8a6f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:53:14.199605 27715 tablet_service.cc:1467] Processing CreateTablet for tablet c525c10d8f0c42cd82c794ba5cff8a6f (DEFAULT_TABLE table=test-table [id=9c6eaa4c8369408c8cce7d3f9908110f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:53:14.201848 27715 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c525c10d8f0c42cd82c794ba5cff8a6f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:53:14.202231 27884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:14.180362 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47826 (local address 127.15.113.130:37171)
0809 22:53:14.187746 (+ 7384us) server_negotiation.cc:206] Beginning negotiation
0809 22:53:14.187755 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 22:53:14.190971 (+ 3216us) server_negotiation.cc:407] Connection header received
0809 22:53:14.191125 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:53:14.191133 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:53:14.191376 (+ 243us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:53:14.191652 (+ 276us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:53:14.193801 (+ 2149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:14.195488 (+ 1687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:14.198895 (+ 3407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:14.200667 (+ 1772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:14.201057 (+ 390us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:14.201225 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0809 22:53:14.201635 (+ 410us) server_negotiation.cc:299] Negotiation successful
0809 22:53:14.201838 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7189,"thread_start_us":6892,"threads_started":1}
I20260809 22:53:14.202265 27882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:14.180106 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.130:37171 (local address 127.0.0.1:47826)
0809 22:53:14.190614 (+ 10508us) negotiation.cc:107] Waiting for socket to connect
0809 22:53:14.190657 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 22:53:14.190834 (+ 177us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:53:14.191979 (+ 1145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:53:14.191983 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:53:14.192010 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:53:14.193621 (+ 1611us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:53:14.193634 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:53:14.195650 (+ 2016us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:53:14.195655 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:53:14.198732 (+ 3077us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:53:14.198749 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:53:14.201093 (+ 2344us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:53:14.201103 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:53:14.201385 (+ 282us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:14.201418 (+ 33us) client_negotiation.cc:769] Sending connection context
0809 22:53:14.201512 (+ 94us) client_negotiation.cc:240] Negotiation successful
0809 22:53:14.201640 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10363,"thread_start_us":10207,"threads_started":1}
I20260809 22:53:14.206887 27489 tablet_service.cc:1467] Processing CreateTablet for tablet c525c10d8f0c42cd82c794ba5cff8a6f (DEFAULT_TABLE table=test-table [id=9c6eaa4c8369408c8cce7d3f9908110f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:53:14.208712 27489 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c525c10d8f0c42cd82c794ba5cff8a6f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:53:14.217357 27886 tablet_bootstrap.cc:492] T c525c10d8f0c42cd82c794ba5cff8a6f P b8e298532adf4449955b1b5dc49552c2: Bootstrap starting.
I20260809 22:53:14.222648 27886 tablet_bootstrap.cc:654] T c525c10d8f0c42cd82c794ba5cff8a6f P b8e298532adf4449955b1b5dc49552c2: Neither blocks nor log segments found. Creating new log.
I20260809 22:53:14.224622 27890 tablet_bootstrap.cc:492] T c525c10d8f0c42cd82c794ba5cff8a6f P a0eed0b7d94e4c1f905abd17c6d0fd80: Bootstrap starting.
I20260809 22:53:14.224852 27886 log.cc:824] T c525c10d8f0c42cd82c794ba5cff8a6f P b8e298532adf4449955b1b5dc49552c2: Log is configured to *not* fsync() on all Append() calls
I20260809 22:53:14.229717 27886 tablet_bootstrap.cc:492] T c525c10d8f0c42cd82c794ba5cff8a6f P b8e298532adf4449955b1b5dc49552c2: No bootstrap required, opened a new log
I20260809 22:53:14.230211 27886 ts_tablet_manager.cc:1397] T c525c10d8f0c42cd82c794ba5cff8a6f P b8e298532adf4449955b1b5dc49552c2: Time spent bootstrapping tablet: real 0.013s user 0.007s sys 0.003s
I20260809 22:53:14.230947 27890 tablet_bootstrap.cc:654] T c525c10d8f0c42cd82c794ba5cff8a6f P a0eed0b7d94e4c1f905abd17c6d0fd80: Neither blocks nor log segments found. Creating new log.
I20260809 22:53:14.231020 27894 tablet_bootstrap.cc:492] T c525c10d8f0c42cd82c794ba5cff8a6f P 200f1dd145df4993b427e20748be2114: Bootstrap starting.
I20260809 22:53:14.233362 27890 log.cc:824] T c525c10d8f0c42cd82c794ba5cff8a6f P a0eed0b7d94e4c1f905abd17c6d0fd80: Log is configured to *not* fsync() on all Append() calls
I20260809 22:53:14.236544 27894 tablet_bootstrap.cc:654] T c525c10d8f0c42cd82c794ba5cff8a6f P 200f1dd145df4993b427e20748be2114: Neither blocks nor log segments found. Creating new log.
I20260809 22:53:14.237663 27890 tablet_bootstrap.cc:492] T c525c10d8f0c42cd82c794ba5cff8a6f P a0eed0b7d94e4c1f905abd17c6d0fd80: No bootstrap required, opened a new log
I20260809 22:53:14.238606 27890 ts_tablet_manager.cc:1397] T c525c10d8f0c42cd82c794ba5cff8a6f P a0eed0b7d94e4c1f905abd17c6d0fd80: Time spent bootstrapping tablet: real 0.014s user 0.008s sys 0.003s
I20260809 22:53:14.239341 27894 log.cc:824] T c525c10d8f0c42cd82c794ba5cff8a6f P 200f1dd145df4993b427e20748be2114: Log is configured to *not* fsync() on all Append() calls
I20260809 22:53:14.244431 27886 raft_consensus.cc:348] T c525c10d8f0c42cd82c794ba5cff8a6f P b8e298532adf4449955b1b5dc49552c2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8e298532adf4449955b1b5dc49552c2" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 38043 } } peers { permanent_uuid: "a0eed0b7d94e4c1f905abd17c6d0fd80" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 32941 } } peers { permanent_uuid: "200f1dd145df4993b427e20748be2114" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 37171 } }
I20260809 22:53:14.245100 27886 raft_consensus.cc:374] T c525c10d8f0c42cd82c794ba5cff8a6f P b8e298532adf4449955b1b5dc49552c2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:53:14.245262 27886 raft_consensus.cc:729] T c525c10d8f0c42cd82c794ba5cff8a6f P b8e298532adf4449955b1b5dc49552c2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8e298532adf4449955b1b5dc49552c2, State: Initialized, Role: FOLLOWER
I20260809 22:53:14.245853 27886 consensus_queue.cc:260] T c525c10d8f0c42cd82c794ba5cff8a6f P b8e298532adf4449955b1b5dc49552c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8e298532adf4449955b1b5dc49552c2" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 38043 } } peers { permanent_uuid: "a0eed0b7d94e4c1f905abd17c6d0fd80" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 32941 } } peers { permanent_uuid: "200f1dd145df4993b427e20748be2114" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 37171 } }
I20260809 22:53:14.246665 27894 tablet_bootstrap.cc:492] T c525c10d8f0c42cd82c794ba5cff8a6f P 200f1dd145df4993b427e20748be2114: No bootstrap required, opened a new log
I20260809 22:53:14.247061 27894 ts_tablet_manager.cc:1397] T c525c10d8f0c42cd82c794ba5cff8a6f P 200f1dd145df4993b427e20748be2114: Time spent bootstrapping tablet: real 0.016s user 0.008s sys 0.005s
I20260809 22:53:14.247701 27886 ts_tablet_manager.cc:1428] T c525c10d8f0c42cd82c794ba5cff8a6f P b8e298532adf4449955b1b5dc49552c2: Time spent starting tablet: real 0.017s user 0.014s sys 0.004s
I20260809 22:53:14.253813 27890 raft_consensus.cc:348] T c525c10d8f0c42cd82c794ba5cff8a6f P a0eed0b7d94e4c1f905abd17c6d0fd80 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8e298532adf4449955b1b5dc49552c2" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 38043 } } peers { permanent_uuid: "a0eed0b7d94e4c1f905abd17c6d0fd80" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 32941 } } peers { permanent_uuid: "200f1dd145df4993b427e20748be2114" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 37171 } }
I20260809 22:53:14.254428 27890 raft_consensus.cc:374] T c525c10d8f0c42cd82c794ba5cff8a6f P a0eed0b7d94e4c1f905abd17c6d0fd80 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:53:14.254591 27890 raft_consensus.cc:729] T c525c10d8f0c42cd82c794ba5cff8a6f P a0eed0b7d94e4c1f905abd17c6d0fd80 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0eed0b7d94e4c1f905abd17c6d0fd80, State: Initialized, Role: FOLLOWER
I20260809 22:53:14.255410 27890 consensus_queue.cc:260] T c525c10d8f0c42cd82c794ba5cff8a6f P a0eed0b7d94e4c1f905abd17c6d0fd80 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8e298532adf4449955b1b5dc49552c2" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 38043 } } peers { permanent_uuid: "a0eed0b7d94e4c1f905abd17c6d0fd80" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 32941 } } peers { permanent_uuid: "200f1dd145df4993b427e20748be2114" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 37171 } }
I20260809 22:53:14.257140 27833 heartbeater.cc:503] Master 127.15.113.190:42225 was elected leader, sending a full tablet report...
I20260809 22:53:14.258103 27890 ts_tablet_manager.cc:1428] T c525c10d8f0c42cd82c794ba5cff8a6f P a0eed0b7d94e4c1f905abd17c6d0fd80: Time spent starting tablet: real 0.019s user 0.012s sys 0.005s
I20260809 22:53:14.265424 27894 raft_consensus.cc:348] T c525c10d8f0c42cd82c794ba5cff8a6f P 200f1dd145df4993b427e20748be2114 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8e298532adf4449955b1b5dc49552c2" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 38043 } } peers { permanent_uuid: "a0eed0b7d94e4c1f905abd17c6d0fd80" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 32941 } } peers { permanent_uuid: "200f1dd145df4993b427e20748be2114" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 37171 } }
I20260809 22:53:14.265884 27894 raft_consensus.cc:374] T c525c10d8f0c42cd82c794ba5cff8a6f P 200f1dd145df4993b427e20748be2114 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:53:14.266031 27894 raft_consensus.cc:729] T c525c10d8f0c42cd82c794ba5cff8a6f P 200f1dd145df4993b427e20748be2114 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 200f1dd145df4993b427e20748be2114, State: Initialized, Role: FOLLOWER
I20260809 22:53:14.266757 27894 consensus_queue.cc:260] T c525c10d8f0c42cd82c794ba5cff8a6f P 200f1dd145df4993b427e20748be2114 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8e298532adf4449955b1b5dc49552c2" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 38043 } } peers { permanent_uuid: "a0eed0b7d94e4c1f905abd17c6d0fd80" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 32941 } } peers { permanent_uuid: "200f1dd145df4993b427e20748be2114" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 37171 } }
I20260809 22:53:14.268782 27593 heartbeater.cc:503] Master 127.15.113.190:42225 was elected leader, sending a full tablet report...
I20260809 22:53:14.269330 27894 ts_tablet_manager.cc:1428] T c525c10d8f0c42cd82c794ba5cff8a6f P 200f1dd145df4993b427e20748be2114: Time spent starting tablet: real 0.022s user 0.009s sys 0.010s
W20260809 22:53:14.310176 27836 tablet.cc:2369] T c525c10d8f0c42cd82c794ba5cff8a6f P a0eed0b7d94e4c1f905abd17c6d0fd80: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 22:53:14.339501 27367 tablet.cc:2369] T c525c10d8f0c42cd82c794ba5cff8a6f P b8e298532adf4449955b1b5dc49552c2: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 22:53:14.426872 27595 tablet.cc:2369] T c525c10d8f0c42cd82c794ba5cff8a6f P 200f1dd145df4993b427e20748be2114: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 22:53:14.485635 27901 raft_consensus.cc:482] T c525c10d8f0c42cd82c794ba5cff8a6f P a0eed0b7d94e4c1f905abd17c6d0fd80 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 22:53:14.486228 27901 raft_consensus.cc:504] T c525c10d8f0c42cd82c794ba5cff8a6f P a0eed0b7d94e4c1f905abd17c6d0fd80 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8e298532adf4449955b1b5dc49552c2" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 38043 } } peers { permanent_uuid: "a0eed0b7d94e4c1f905abd17c6d0fd80" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 32941 } } peers { permanent_uuid: "200f1dd145df4993b427e20748be2114" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 37171 } }
I20260809 22:53:14.488471 27901 leader_election.cc:288] T c525c10d8f0c42cd82c794ba5cff8a6f P a0eed0b7d94e4c1f905abd17c6d0fd80 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b8e298532adf4449955b1b5dc49552c2 (127.15.113.129:38043), 200f1dd145df4993b427e20748be2114 (127.15.113.130:37171)
I20260809 22:53:14.496263 27878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:14.488846 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.113.131:56713 (local address 127.15.113.129:38043)
0809 22:53:14.489089 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0809 22:53:14.489095 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:53:14.489368 (+ 273us) server_negotiation.cc:407] Connection header received
0809 22:53:14.489625 (+ 257us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:53:14.489630 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:53:14.489727 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:53:14.489813 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:53:14.490317 (+ 504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:14.491256 (+ 939us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:14.493828 (+ 2572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:14.495092 (+ 1264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:14.495231 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:14.495310 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0809 22:53:14.495730 (+ 420us) server_negotiation.cc:299] Negotiation successful
0809 22:53:14.495855 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260809 22:53:14.496192 27843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:14.489130 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.129:38043 (local address 127.15.113.131:56713)
0809 22:53:14.489278 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0809 22:53:14.489295 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 22:53:14.489424 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:53:14.489846 (+ 422us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:53:14.489851 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:53:14.489878 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:53:14.490180 (+ 302us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:53:14.490190 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:53:14.491421 (+ 1231us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:53:14.491425 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:53:14.493692 (+ 2267us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:53:14.493705 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:53:14.495297 (+ 1592us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:53:14.495303 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:53:14.495573 (+ 270us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:14.495598 (+ 25us) client_negotiation.cc:769] Sending connection context
0809 22:53:14.495682 (+ 84us) client_negotiation.cc:240] Negotiation successful
0809 22:53:14.495829 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":48,"mutex_wait_us":21}
I20260809 22:53:14.496595 27884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:14.489792 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.113.131:36077 (local address 127.15.113.130:37171)
0809 22:53:14.490041 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0809 22:53:14.490045 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 22:53:14.490060 (+ 15us) server_negotiation.cc:407] Connection header received
0809 22:53:14.490117 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:53:14.490120 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:53:14.490172 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:53:14.490250 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:53:14.490679 (+ 429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:14.491629 (+ 950us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:14.494105 (+ 2476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:14.495328 (+ 1223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:14.495508 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:14.495588 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0809 22:53:14.496040 (+ 452us) server_negotiation.cc:299] Negotiation successful
0809 22:53:14.496169 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260809 22:53:14.497242 27839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:14.489587 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.130:37171 (local address 127.15.113.131:36077)
0809 22:53:14.489748 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0809 22:53:14.489764 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 22:53:14.489852 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:53:14.490294 (+ 442us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:53:14.490299 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:53:14.490324 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:53:14.490541 (+ 217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:53:14.490552 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:53:14.491819 (+ 1267us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:53:14.491823 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:53:14.493955 (+ 2132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:53:14.493969 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:53:14.495486 (+ 1517us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:53:14.495492 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:53:14.495884 (+ 392us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:14.495910 (+ 26us) client_negotiation.cc:769] Sending connection context
0809 22:53:14.495993 (+ 83us) client_negotiation.cc:240] Negotiation successful
0809 22:53:14.496169 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40,"mutex_wait_us":29}
I20260809 22:53:14.497885 27271 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c525c10d8f0c42cd82c794ba5cff8a6f" candidate_uuid: "a0eed0b7d94e4c1f905abd17c6d0fd80" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8e298532adf4449955b1b5dc49552c2" is_pre_election: true
I20260809 22:53:14.498589 27521 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c525c10d8f0c42cd82c794ba5cff8a6f" candidate_uuid: "a0eed0b7d94e4c1f905abd17c6d0fd80" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "200f1dd145df4993b427e20748be2114" is_pre_election: true
I20260809 22:53:14.498828 27271 raft_consensus.cc:2454] T c525c10d8f0c42cd82c794ba5cff8a6f P b8e298532adf4449955b1b5dc49552c2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0eed0b7d94e4c1f905abd17c6d0fd80 in term 0.
I20260809 22:53:14.499410 27521 raft_consensus.cc:2454] T c525c10d8f0c42cd82c794ba5cff8a6f P 200f1dd145df4993b427e20748be2114 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0eed0b7d94e4c1f905abd17c6d0fd80 in term 0.
I20260809 22:53:14.500062 27641 leader_election.cc:302] T c525c10d8f0c42cd82c794ba5cff8a6f P a0eed0b7d94e4c1f905abd17c6d0fd80 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a0eed0b7d94e4c1f905abd17c6d0fd80, b8e298532adf4449955b1b5dc49552c2; no voters:
I20260809 22:53:14.500415 27901 raft_consensus.cc:2781] T c525c10d8f0c42cd82c794ba5cff8a6f P a0eed0b7d94e4c1f905abd17c6d0fd80 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 22:53:14.500552 27901 raft_consensus.cc:482] T c525c10d8f0c42cd82c794ba5cff8a6f P a0eed0b7d94e4c1f905abd17c6d0fd80 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 22:53:14.500660 27901 raft_consensus.cc:3037] T c525c10d8f0c42cd82c794ba5cff8a6f P a0eed0b7d94e4c1f905abd17c6d0fd80 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:53:14.504033 27901 raft_consensus.cc:504] T c525c10d8f0c42cd82c794ba5cff8a6f P a0eed0b7d94e4c1f905abd17c6d0fd80 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8e298532adf4449955b1b5dc49552c2" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 38043 } } peers { permanent_uuid: "a0eed0b7d94e4c1f905abd17c6d0fd80" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 32941 } } peers { permanent_uuid: "200f1dd145df4993b427e20748be2114" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 37171 } }
I20260809 22:53:14.504778 27901 leader_election.cc:288] T c525c10d8f0c42cd82c794ba5cff8a6f P a0eed0b7d94e4c1f905abd17c6d0fd80 [CANDIDATE]: Term 1 election: Requested vote from peers b8e298532adf4449955b1b5dc49552c2 (127.15.113.129:38043), 200f1dd145df4993b427e20748be2114 (127.15.113.130:37171)
I20260809 22:53:14.505379 27521 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c525c10d8f0c42cd82c794ba5cff8a6f" candidate_uuid: "a0eed0b7d94e4c1f905abd17c6d0fd80" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "200f1dd145df4993b427e20748be2114"
I20260809 22:53:14.505450 27271 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c525c10d8f0c42cd82c794ba5cff8a6f" candidate_uuid: "a0eed0b7d94e4c1f905abd17c6d0fd80" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8e298532adf4449955b1b5dc49552c2"
I20260809 22:53:14.505589 27521 raft_consensus.cc:3037] T c525c10d8f0c42cd82c794ba5cff8a6f P 200f1dd145df4993b427e20748be2114 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:53:14.505738 27271 raft_consensus.cc:3037] T c525c10d8f0c42cd82c794ba5cff8a6f P b8e298532adf4449955b1b5dc49552c2 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:53:14.509354 27521 raft_consensus.cc:2454] T c525c10d8f0c42cd82c794ba5cff8a6f P 200f1dd145df4993b427e20748be2114 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0eed0b7d94e4c1f905abd17c6d0fd80 in term 1.
I20260809 22:53:14.509521 27271 raft_consensus.cc:2454] T c525c10d8f0c42cd82c794ba5cff8a6f P b8e298532adf4449955b1b5dc49552c2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0eed0b7d94e4c1f905abd17c6d0fd80 in term 1.
I20260809 22:53:14.509856 27641 leader_election.cc:302] T c525c10d8f0c42cd82c794ba5cff8a6f P a0eed0b7d94e4c1f905abd17c6d0fd80 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 200f1dd145df4993b427e20748be2114, a0eed0b7d94e4c1f905abd17c6d0fd80; no voters:
I20260809 22:53:14.510267 27901 raft_consensus.cc:2781] T c525c10d8f0c42cd82c794ba5cff8a6f P a0eed0b7d94e4c1f905abd17c6d0fd80 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 22:53:14.511098 27901 raft_consensus.cc:686] T c525c10d8f0c42cd82c794ba5cff8a6f P a0eed0b7d94e4c1f905abd17c6d0fd80 [term 1 LEADER]: Becoming Leader. State: Replica: a0eed0b7d94e4c1f905abd17c6d0fd80, State: Running, Role: LEADER
I20260809 22:53:14.511758 27901 consensus_queue.cc:237] T c525c10d8f0c42cd82c794ba5cff8a6f P a0eed0b7d94e4c1f905abd17c6d0fd80 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8e298532adf4449955b1b5dc49552c2" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 38043 } } peers { permanent_uuid: "a0eed0b7d94e4c1f905abd17c6d0fd80" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 32941 } } peers { permanent_uuid: "200f1dd145df4993b427e20748be2114" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 37171 } }
I20260809 22:53:14.518432 25770 catalog_manager.cc:5495] T c525c10d8f0c42cd82c794ba5cff8a6f P a0eed0b7d94e4c1f905abd17c6d0fd80 reported cstate change: term changed from 0 to 1, leader changed from <none> to a0eed0b7d94e4c1f905abd17c6d0fd80 (127.15.113.131). New cstate: current_term: 1 leader_uuid: "a0eed0b7d94e4c1f905abd17c6d0fd80" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8e298532adf4449955b1b5dc49552c2" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 38043 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a0eed0b7d94e4c1f905abd17c6d0fd80" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 32941 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "200f1dd145df4993b427e20748be2114" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 37171 } health_report { overall_health: UNKNOWN } } }
I20260809 22:53:14.566776 27881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:14.559437 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58120 (local address 127.15.113.131:32941)
0809 22:53:14.559734 (+ 297us) server_negotiation.cc:206] Beginning negotiation
0809 22:53:14.559741 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:53:14.560161 (+ 420us) server_negotiation.cc:407] Connection header received
0809 22:53:14.560339 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:53:14.560343 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:53:14.560490 (+ 147us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:53:14.560638 (+ 148us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 22:53:14.561251 (+ 613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:14.562334 (+ 1083us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:14.564107 (+ 1773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:14.564847 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:14.565000 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:14.565603 (+ 603us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 22:53:14.566152 (+ 549us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 22:53:14.566305 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0809 22:53:14.566344 (+ 39us) server_negotiation.cc:299] Negotiation successful
0809 22:53:14.566409 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260809 22:53:14.577210 27521 raft_consensus.cc:1261] T c525c10d8f0c42cd82c794ba5cff8a6f P 200f1dd145df4993b427e20748be2114 [term 1 FOLLOWER]: Refusing update from remote peer a0eed0b7d94e4c1f905abd17c6d0fd80: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 22:53:14.577481 27271 raft_consensus.cc:1261] T c525c10d8f0c42cd82c794ba5cff8a6f P b8e298532adf4449955b1b5dc49552c2 [term 1 FOLLOWER]: Refusing update from remote peer a0eed0b7d94e4c1f905abd17c6d0fd80: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 22:53:14.578382 27906 consensus_queue.cc:1035] T c525c10d8f0c42cd82c794ba5cff8a6f P a0eed0b7d94e4c1f905abd17c6d0fd80 [LEADER]: Connected to new peer: Peer: permanent_uuid: "200f1dd145df4993b427e20748be2114" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 37171 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 22:53:14.578866 27901 consensus_queue.cc:1035] T c525c10d8f0c42cd82c794ba5cff8a6f P a0eed0b7d94e4c1f905abd17c6d0fd80 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8e298532adf4449955b1b5dc49552c2" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 38043 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 22:53:14.594772 27915 mvcc.cc:205] Tried to move back new op lower bound from 7316750313778028544 to 7316750313524645888. Current Snapshot: MvccSnapshot[applied={T|T < 7316750313778028544}]
I20260809 22:53:14.598583 27911 mvcc.cc:205] Tried to move back new op lower bound from 7316750313778028544 to 7316750313524645888. Current Snapshot: MvccSnapshot[applied={T|T < 7316750313778028544 or (T in {7316750313778028544})}]
I20260809 22:53:14.620247 25770 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52230:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 22:53:14.620653 25770 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52230:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 22:53:14.627818 25770 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 460ec5412d65461a811d766224ae4ab6: Sending DeleteTablet for 3 replicas of tablet c525c10d8f0c42cd82c794ba5cff8a6f
I20260809 22:53:14.629395 27489 tablet_service.cc:1514] Processing DeleteTablet for tablet c525c10d8f0c42cd82c794ba5cff8a6f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:53:14 UTC) from {username='oryx', principal='oryx/127.15.113.190@KRBTEST.COM'} at 127.0.0.1:47826
I20260809 22:53:14.629478 27715 tablet_service.cc:1514] Processing DeleteTablet for tablet c525c10d8f0c42cd82c794ba5cff8a6f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:53:14 UTC) from {username='oryx', principal='oryx/127.15.113.190@KRBTEST.COM'} at 127.0.0.1:58110
I20260809 22:53:14.629631 27236 tablet_service.cc:1514] Processing DeleteTablet for tablet c525c10d8f0c42cd82c794ba5cff8a6f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:53:14 UTC) from {username='oryx', principal='oryx/127.15.113.190@KRBTEST.COM'} at 127.0.0.1:39810
I20260809 22:53:14.639602 27932 tablet_replica.cc:330] T c525c10d8f0c42cd82c794ba5cff8a6f P a0eed0b7d94e4c1f905abd17c6d0fd80: stopping tablet replica
I20260809 22:53:14.640208 27932 raft_consensus.cc:2229] T c525c10d8f0c42cd82c794ba5cff8a6f P a0eed0b7d94e4c1f905abd17c6d0fd80 [term 1 LEADER]: Raft consensus shutting down.
I20260809 22:53:14.641179 27932 raft_consensus.cc:2258] T c525c10d8f0c42cd82c794ba5cff8a6f P a0eed0b7d94e4c1f905abd17c6d0fd80 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 22:53:14.641570 27934 tablet_replica.cc:330] T c525c10d8f0c42cd82c794ba5cff8a6f P b8e298532adf4449955b1b5dc49552c2: stopping tablet replica
I20260809 22:53:14.642170 27934 raft_consensus.cc:2229] T c525c10d8f0c42cd82c794ba5cff8a6f P b8e298532adf4449955b1b5dc49552c2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 22:53:14.643173 27934 raft_consensus.cc:2258] T c525c10d8f0c42cd82c794ba5cff8a6f P b8e298532adf4449955b1b5dc49552c2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 22:53:14.646243 27932 ts_tablet_manager.cc:1905] T c525c10d8f0c42cd82c794ba5cff8a6f P a0eed0b7d94e4c1f905abd17c6d0fd80: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 22:53:14.647339 15814 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskItYPJR/build/asan/bin/kudu pid 25717
I20260809 22:53:14.648532 25825 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:53:14.651762 27934 ts_tablet_manager.cc:1905] T c525c10d8f0c42cd82c794ba5cff8a6f P b8e298532adf4449955b1b5dc49552c2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 22:53:14.656569 27932 ts_tablet_manager.cc:1918] T c525c10d8f0c42cd82c794ba5cff8a6f P a0eed0b7d94e4c1f905abd17c6d0fd80: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 22:53:14.656924 27932 log.cc:1196] T c525c10d8f0c42cd82c794ba5cff8a6f P a0eed0b7d94e4c1f905abd17c6d0fd80: Deleting WAL directory at /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315935617919-15814-0/minicluster-data/ts-2/wal/wals/c525c10d8f0c42cd82c794ba5cff8a6f
I20260809 22:53:14.657265 27931 tablet_replica.cc:330] T c525c10d8f0c42cd82c794ba5cff8a6f P 200f1dd145df4993b427e20748be2114: stopping tablet replica
I20260809 22:53:14.658183 27932 ts_tablet_manager.cc:1939] T c525c10d8f0c42cd82c794ba5cff8a6f P a0eed0b7d94e4c1f905abd17c6d0fd80: Deleting consensus metadata
I20260809 22:53:14.658231 27931 raft_consensus.cc:2229] T c525c10d8f0c42cd82c794ba5cff8a6f P 200f1dd145df4993b427e20748be2114 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 22:53:14.660650 27931 raft_consensus.cc:2258] T c525c10d8f0c42cd82c794ba5cff8a6f P 200f1dd145df4993b427e20748be2114 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 22:53:14.663758 27931 ts_tablet_manager.cc:1905] T c525c10d8f0c42cd82c794ba5cff8a6f P 200f1dd145df4993b427e20748be2114: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 22:53:14.669826 27934 ts_tablet_manager.cc:1918] T c525c10d8f0c42cd82c794ba5cff8a6f P b8e298532adf4449955b1b5dc49552c2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 22:53:14.670089 27934 log.cc:1196] T c525c10d8f0c42cd82c794ba5cff8a6f P b8e298532adf4449955b1b5dc49552c2: Deleting WAL directory at /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315935617919-15814-0/minicluster-data/ts-0/wal/wals/c525c10d8f0c42cd82c794ba5cff8a6f
I20260809 22:53:14.670807 27934 ts_tablet_manager.cc:1939] T c525c10d8f0c42cd82c794ba5cff8a6f P b8e298532adf4449955b1b5dc49552c2: Deleting consensus metadata
I20260809 22:53:14.676313 27856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:14.633765 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50752 (local address 127.15.113.189:41115)
0809 22:53:14.633974 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0809 22:53:14.633982 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 22:53:14.639538 (+ 5556us) server_negotiation.cc:407] Connection header received
0809 22:53:14.639934 (+ 396us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:53:14.639940 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:53:14.640073 (+ 133us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:53:14.640151 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:53:14.645032 (+ 4881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:14.646428 (+ 1396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:14.661928 (+ 15500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:14.662667 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:14.662802 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:14.667450 (+ 4648us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:53:14.667458 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:53:14.667474 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:53:14.667517 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:53:14.672471 (+ 4954us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:53:14.673639 (+ 1168us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:53:14.673644 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:53:14.673648 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:53:14.673895 (+ 247us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:53:14.674470 (+ 575us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:53:14.674476 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:53:14.674480 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:53:14.674898 (+ 418us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:53:14.675165 (+ 267us) server_negotiation.cc:1035] Waiting for connection context
0809 22:53:14.675737 (+ 572us) server_negotiation.cc:299] Negotiation successful
0809 22:53:14.675968 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260809 22:53:14.680528 27931 ts_tablet_manager.cc:1918] T c525c10d8f0c42cd82c794ba5cff8a6f P 200f1dd145df4993b427e20748be2114: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 22:53:14.680761 27931 log.cc:1196] T c525c10d8f0c42cd82c794ba5cff8a6f P 200f1dd145df4993b427e20748be2114: Deleting WAL directory at /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315935617919-15814-0/minicluster-data/ts-1/wal/wals/c525c10d8f0c42cd82c794ba5cff8a6f
I20260809 22:53:14.681725 27931 ts_tablet_manager.cc:1939] T c525c10d8f0c42cd82c794ba5cff8a6f P 200f1dd145df4993b427e20748be2114: Deleting consensus metadata
I20260809 22:53:14.701947 27936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:14.644985 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37824 (local address 127.15.113.188:43669)
0809 22:53:14.665941 (+ 20956us) server_negotiation.cc:206] Beginning negotiation
0809 22:53:14.665948 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:53:14.679368 (+ 13420us) server_negotiation.cc:407] Connection header received
0809 22:53:14.679607 (+ 239us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:53:14.679613 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:53:14.679691 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:53:14.679760 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:53:14.682313 (+ 2553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:14.683854 (+ 1541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:14.686519 (+ 2665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:14.687435 (+ 916us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:14.687583 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:14.692992 (+ 5409us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:53:14.693002 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:53:14.693007 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:53:14.693052 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:53:14.698376 (+ 5324us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:53:14.699166 (+ 790us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:53:14.699172 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:53:14.699176 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:53:14.699350 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:53:14.699921 (+ 571us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:53:14.699926 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:53:14.699930 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:53:14.700303 (+ 373us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:53:14.700653 (+ 350us) server_negotiation.cc:1035] Waiting for connection context
0809 22:53:14.701152 (+ 499us) server_negotiation.cc:299] Negotiation successful
0809 22:53:14.701388 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20782,"thread_start_us":21078,"threads_started":1}
I20260809 22:53:14.798840 25749 catalog_manager.cc:4841] TS a0eed0b7d94e4c1f905abd17c6d0fd80 (127.15.113.131:32941): tablet c525c10d8f0c42cd82c794ba5cff8a6f (table test-table [id=9c6eaa4c8369408c8cce7d3f9908110f]) successfully deleted
W20260809 22:53:14.799203 25749 outbound_call.cc:322] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.15.113.131:32941, user_credentials={real_user=slave}} blocked reactor thread for 138746us
I20260809 22:53:14.800182 25750 catalog_manager.cc:4841] TS 200f1dd145df4993b427e20748be2114 (127.15.113.130:37171): tablet c525c10d8f0c42cd82c794ba5cff8a6f (table test-table [id=9c6eaa4c8369408c8cce7d3f9908110f]) successfully deleted
I20260809 22:53:14.800457 25750 catalog_manager.cc:4841] TS b8e298532adf4449955b1b5dc49552c2 (127.15.113.129:38043): tablet c525c10d8f0c42cd82c794ba5cff8a6f (table test-table [id=9c6eaa4c8369408c8cce7d3f9908110f]) successfully deleted
I20260809 22:53:14.801019 15814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskItYPJR/build/asan/bin/kudu with pid 25717
I20260809 22:53:14.863662 27856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:14.843369 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50764 (local address 127.15.113.189:41115)
0809 22:53:14.843544 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0809 22:53:14.843549 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:53:14.845539 (+ 1990us) server_negotiation.cc:407] Connection header received
0809 22:53:14.845730 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:53:14.845736 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:53:14.845830 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:53:14.845933 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:53:14.848093 (+ 2160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:14.849574 (+ 1481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:14.851942 (+ 2368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:14.852584 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:14.852715 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:14.856327 (+ 3612us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:53:14.856337 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:53:14.856343 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:53:14.856392 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:53:14.859906 (+ 3514us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:53:14.861273 (+ 1367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:53:14.861279 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:53:14.861282 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:53:14.861643 (+ 361us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:53:14.862125 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:53:14.862131 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:53:14.862134 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:53:14.862448 (+ 314us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:53:14.862701 (+ 253us) server_negotiation.cc:1035] Waiting for connection context
0809 22:53:14.863075 (+ 374us) server_negotiation.cc:299] Negotiation successful
0809 22:53:14.863303 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260809 22:53:14.865305 27936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:14.844883 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37826 (local address 127.15.113.188:43669)
0809 22:53:14.845170 (+ 287us) server_negotiation.cc:206] Beginning negotiation
0809 22:53:14.845176 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:53:14.848408 (+ 3232us) server_negotiation.cc:407] Connection header received
0809 22:53:14.848567 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:53:14.848572 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:53:14.848653 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:53:14.848777 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:53:14.851011 (+ 2234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:14.852247 (+ 1236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:14.853842 (+ 1595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:14.855005 (+ 1163us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:14.855111 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:14.858566 (+ 3455us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:53:14.858573 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:53:14.858578 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:53:14.858614 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:53:14.861608 (+ 2994us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:53:14.862382 (+ 774us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:53:14.862387 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:53:14.862390 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:53:14.862632 (+ 242us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:53:14.863242 (+ 610us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:53:14.863248 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:53:14.863252 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:53:14.863731 (+ 479us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:53:14.864037 (+ 306us) server_negotiation.cc:1035] Waiting for connection context
0809 22:53:14.864840 (+ 803us) server_negotiation.cc:299] Negotiation successful
0809 22:53:14.865088 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260809 22:53:15.607342 27955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:15.606295 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:42225 (local address 127.15.113.131:34095)
0809 22:53:15.606977 (+ 682us) negotiation.cc:107] Waiting for socket to connect
0809 22:53:15.607121 (+ 144us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.113.190:42225: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":551,"thread_start_us":430,"threads_started":1}
W20260809 22:53:15.607986 27833 heartbeater.cc:644] Failed to heartbeat to 127.15.113.190:42225 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.15.113.190:42225: connect: Connection refused (error 111)
I20260809 22:53:15.608799 27956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:15.608099 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:42225 (local address 127.15.113.129:38081)
0809 22:53:15.608544 (+ 445us) negotiation.cc:107] Waiting for socket to connect
0809 22:53:15.608660 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.113.190:42225: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":318,"thread_start_us":193,"threads_started":1}
W20260809 22:53:15.609244 27353 heartbeater.cc:644] Failed to heartbeat to 127.15.113.190:42225 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.15.113.190:42225: connect: Connection refused (error 111)
I20260809 22:53:15.613879 27957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:15.613176 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:42225 (local address 127.15.113.130:43965)
0809 22:53:15.613556 (+ 380us) negotiation.cc:107] Waiting for socket to connect
0809 22:53:15.613690 (+ 134us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.113.190:42225: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":288,"thread_start_us":180,"threads_started":1}
W20260809 22:53:15.614346 27593 heartbeater.cc:644] Failed to heartbeat to 127.15.113.190:42225 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.15.113.190:42225: connect: Connection refused (error 111)
I20260809 22:53:16.128921 27961 raft_consensus.cc:482] T 00000000000000000000000000000000 P 70317072228d4a47877776b7fb4e21d1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 460ec5412d65461a811d766224ae4ab6)
I20260809 22:53:16.129078 27964 raft_consensus.cc:482] T 00000000000000000000000000000000 P 87d4ebb338564421816a43cffeb42715 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 460ec5412d65461a811d766224ae4ab6)
I20260809 22:53:16.129235 27961 raft_consensus.cc:504] T 00000000000000000000000000000000 P 70317072228d4a47877776b7fb4e21d1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "460ec5412d65461a811d766224ae4ab6" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 42225 } } peers { permanent_uuid: "87d4ebb338564421816a43cffeb42715" member_type: VOTER last_known_addr { host: "127.15.113.189" port: 41115 } attrs { promote: false } } peers { permanent_uuid: "70317072228d4a47877776b7fb4e21d1" member_type: VOTER last_known_addr { host: "127.15.113.188" port: 43669 } attrs { promote: false } }
I20260809 22:53:16.129398 27964 raft_consensus.cc:504] T 00000000000000000000000000000000 P 87d4ebb338564421816a43cffeb42715 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "460ec5412d65461a811d766224ae4ab6" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 42225 } } peers { permanent_uuid: "87d4ebb338564421816a43cffeb42715" member_type: VOTER last_known_addr { host: "127.15.113.189" port: 41115 } attrs { promote: false } } peers { permanent_uuid: "70317072228d4a47877776b7fb4e21d1" member_type: VOTER last_known_addr { host: "127.15.113.188" port: 43669 } attrs { promote: false } }
I20260809 22:53:16.132172 27965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:16.131041 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:42225 (local address 127.0.0.1:52700)
0809 22:53:16.131713 (+ 672us) negotiation.cc:107] Waiting for socket to connect
0809 22:53:16.131921 (+ 208us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.113.190:42225: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":554,"thread_start_us":426,"threads_started":1}
I20260809 22:53:16.133091 27964 leader_election.cc:288] T 00000000000000000000000000000000 P 87d4ebb338564421816a43cffeb42715 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 460ec5412d65461a811d766224ae4ab6 (127.15.113.190:42225), 70317072228d4a47877776b7fb4e21d1 (127.15.113.188:43669)
I20260809 22:53:16.133616 27961 leader_election.cc:288] T 00000000000000000000000000000000 P 70317072228d4a47877776b7fb4e21d1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 460ec5412d65461a811d766224ae4ab6 (127.15.113.190:42225), 87d4ebb338564421816a43cffeb42715 (127.15.113.189:41115)
I20260809 22:53:16.134142 27967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:16.131088 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:42225 (local address 127.0.0.1:52716)
0809 22:53:16.133609 (+ 2521us) negotiation.cc:107] Waiting for socket to connect
0809 22:53:16.133825 (+ 216us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.113.190:42225: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2391,"thread_start_us":2800,"threads_started":1}
W20260809 22:53:16.134452 27031 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.15.113.190:42225: connect: Connection refused (error 111)
W20260809 22:53:16.134697 26720 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.15.113.190:42225: connect: Connection refused (error 111)
I20260809 22:53:16.136370 27968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:16.135762 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:42225 (local address 127.0.0.1:52728)
0809 22:53:16.136004 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0809 22:53:16.136220 (+ 216us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.113.190:42225: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108}
W20260809 22:53:16.136554 26720 leader_election.cc:334] T 00000000000000000000000000000000 P 87d4ebb338564421816a43cffeb42715 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 460ec5412d65461a811d766224ae4ab6 (127.15.113.190:42225): Network error: Client connection negotiation failed: client connection to 127.15.113.190:42225: connect: Connection refused (error 111)
I20260809 22:53:16.136780 27967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:16.136319 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:42225 (local address 127.0.0.1:52740)
0809 22:53:16.136490 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0809 22:53:16.136593 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.113.190:42225: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
W20260809 22:53:16.138094 27031 leader_election.cc:334] T 00000000000000000000000000000000 P 70317072228d4a47877776b7fb4e21d1 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 460ec5412d65461a811d766224ae4ab6 (127.15.113.190:42225): Network error: Client connection negotiation failed: client connection to 127.15.113.190:42225: connect: Connection refused (error 111)
I20260809 22:53:16.141793 27965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:16.131955 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.188:43669 (local address 127.0.0.1:37830)
0809 22:53:16.132360 (+ 405us) negotiation.cc:107] Waiting for socket to connect
0809 22:53:16.132377 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 22:53:16.132511 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:53:16.135756 (+ 3245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:53:16.135760 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:53:16.135786 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:53:16.136052 (+ 266us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:53:16.136064 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:53:16.137998 (+ 1934us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:53:16.138003 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:53:16.139963 (+ 1960us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:53:16.139979 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:53:16.141094 (+ 1115us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:53:16.141100 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:53:16.141273 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:16.141297 (+ 24us) client_negotiation.cc:769] Sending connection context
0809 22:53:16.141379 (+ 82us) client_negotiation.cc:240] Negotiation successful
0809 22:53:16.141516 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":322,"thread_start_us":2475,"threads_started":1}
I20260809 22:53:16.142069 27969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:16.132396 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37830 (local address 127.15.113.188:43669)
0809 22:53:16.135446 (+ 3050us) server_negotiation.cc:206] Beginning negotiation
0809 22:53:16.135453 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:53:16.135470 (+ 17us) server_negotiation.cc:407] Connection header received
0809 22:53:16.135522 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:53:16.135528 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:53:16.135594 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:53:16.136223 (+ 629us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:53:16.136555 (+ 332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:16.137847 (+ 1292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:16.140111 (+ 2264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:16.140955 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:16.141090 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:16.141174 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0809 22:53:16.141418 (+ 244us) server_negotiation.cc:299] Negotiation successful
0809 22:53:16.141544 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2924,"thread_start_us":2932,"threads_started":1}
I20260809 22:53:16.142158 27966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:16.131201 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.189:41115 (local address 127.0.0.1:50778)
0809 22:53:16.134172 (+ 2971us) negotiation.cc:107] Waiting for socket to connect
0809 22:53:16.134194 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0809 22:53:16.134359 (+ 165us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:53:16.134958 (+ 599us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:53:16.134963 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:53:16.134998 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:53:16.136110 (+ 1112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:53:16.136470 (+ 360us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:53:16.138430 (+ 1960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:53:16.138435 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:53:16.140340 (+ 1905us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:53:16.140353 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:53:16.141465 (+ 1112us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:53:16.141472 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:53:16.141607 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:16.141643 (+ 36us) client_negotiation.cc:769] Sending connection context
0809 22:53:16.141722 (+ 79us) client_negotiation.cc:240] Negotiation successful
0809 22:53:16.141845 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2830,"thread_start_us":2145,"threads_started":1}
I20260809 22:53:16.142220 27970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:16.132479 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50778 (local address 127.15.113.189:41115)
0809 22:53:16.133014 (+ 535us) server_negotiation.cc:206] Beginning negotiation
0809 22:53:16.133020 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:53:16.134267 (+ 1247us) server_negotiation.cc:407] Connection header received
0809 22:53:16.134529 (+ 262us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:53:16.134534 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:53:16.134615 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:53:16.134690 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:53:16.136697 (+ 2007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:16.137924 (+ 1227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:16.140475 (+ 2551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:16.141280 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:16.141441 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:16.141507 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0809 22:53:16.141774 (+ 267us) server_negotiation.cc:299] Negotiation successful
0809 22:53:16.141907 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":399,"thread_start_us":414,"threads_started":1}
I20260809 22:53:16.142584 27070 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "87d4ebb338564421816a43cffeb42715" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "70317072228d4a47877776b7fb4e21d1" is_pre_election: true
I20260809 22:53:16.142765 26762 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "70317072228d4a47877776b7fb4e21d1" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "87d4ebb338564421816a43cffeb42715" is_pre_election: true
I20260809 22:53:16.142882 27070 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 70317072228d4a47877776b7fb4e21d1 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 87d4ebb338564421816a43cffeb42715 in term 1.
I20260809 22:53:16.143035 26762 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 87d4ebb338564421816a43cffeb42715 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70317072228d4a47877776b7fb4e21d1 in term 1.
I20260809 22:53:16.143352 26720 leader_election.cc:302] T 00000000000000000000000000000000 P 87d4ebb338564421816a43cffeb42715 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 70317072228d4a47877776b7fb4e21d1, 87d4ebb338564421816a43cffeb42715; no voters: 460ec5412d65461a811d766224ae4ab6
I20260809 22:53:16.143440 27038 leader_election.cc:302] T 00000000000000000000000000000000 P 70317072228d4a47877776b7fb4e21d1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 70317072228d4a47877776b7fb4e21d1, 87d4ebb338564421816a43cffeb42715; no voters: 460ec5412d65461a811d766224ae4ab6
I20260809 22:53:16.143664 27964 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 87d4ebb338564421816a43cffeb42715 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260809 22:53:16.143791 27964 raft_consensus.cc:482] T 00000000000000000000000000000000 P 87d4ebb338564421816a43cffeb42715 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 460ec5412d65461a811d766224ae4ab6)
I20260809 22:53:16.143827 27961 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 70317072228d4a47877776b7fb4e21d1 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260809 22:53:16.143901 27964 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 87d4ebb338564421816a43cffeb42715 [term 1 FOLLOWER]: Advancing to term 2
I20260809 22:53:16.143945 27961 raft_consensus.cc:482] T 00000000000000000000000000000000 P 70317072228d4a47877776b7fb4e21d1 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 460ec5412d65461a811d766224ae4ab6)
I20260809 22:53:16.144042 27961 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 70317072228d4a47877776b7fb4e21d1 [term 1 FOLLOWER]: Advancing to term 2
I20260809 22:53:16.146890 27961 raft_consensus.cc:504] T 00000000000000000000000000000000 P 70317072228d4a47877776b7fb4e21d1 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "460ec5412d65461a811d766224ae4ab6" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 42225 } } peers { permanent_uuid: "87d4ebb338564421816a43cffeb42715" member_type: VOTER last_known_addr { host: "127.15.113.189" port: 41115 } attrs { promote: false } } peers { permanent_uuid: "70317072228d4a47877776b7fb4e21d1" member_type: VOTER last_known_addr { host: "127.15.113.188" port: 43669 } attrs { promote: false } }
I20260809 22:53:16.146962 27964 raft_consensus.cc:504] T 00000000000000000000000000000000 P 87d4ebb338564421816a43cffeb42715 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "460ec5412d65461a811d766224ae4ab6" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 42225 } } peers { permanent_uuid: "87d4ebb338564421816a43cffeb42715" member_type: VOTER last_known_addr { host: "127.15.113.189" port: 41115 } attrs { promote: false } } peers { permanent_uuid: "70317072228d4a47877776b7fb4e21d1" member_type: VOTER last_known_addr { host: "127.15.113.188" port: 43669 } attrs { promote: false } }
I20260809 22:53:16.147486 27961 leader_election.cc:288] T 00000000000000000000000000000000 P 70317072228d4a47877776b7fb4e21d1 [CANDIDATE]: Term 2 election: Requested vote from peers 460ec5412d65461a811d766224ae4ab6 (127.15.113.190:42225), 87d4ebb338564421816a43cffeb42715 (127.15.113.189:41115)
I20260809 22:53:16.147568 27964 leader_election.cc:288] T 00000000000000000000000000000000 P 87d4ebb338564421816a43cffeb42715 [CANDIDATE]: Term 2 election: Requested vote from peers 460ec5412d65461a811d766224ae4ab6 (127.15.113.190:42225), 70317072228d4a47877776b7fb4e21d1 (127.15.113.188:43669)
I20260809 22:53:16.147990 27966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:16.147576 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:42225 (local address 127.0.0.1:52754)
0809 22:53:16.147787 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0809 22:53:16.147863 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.113.190:42225: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110}
I20260809 22:53:16.147912 26762 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "70317072228d4a47877776b7fb4e21d1" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "87d4ebb338564421816a43cffeb42715"
I20260809 22:53:16.148188 26762 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 87d4ebb338564421816a43cffeb42715 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 70317072228d4a47877776b7fb4e21d1 in current term 2: Already voted for candidate 87d4ebb338564421816a43cffeb42715 in this term.
I20260809 22:53:16.148542 27965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:16.147915 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:42225 (local address 127.0.0.1:52770)
0809 22:53:16.148068 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0809 22:53:16.148166 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.113.190:42225: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260809 22:53:16.148545 27070 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "87d4ebb338564421816a43cffeb42715" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "70317072228d4a47877776b7fb4e21d1"
I20260809 22:53:16.148842 27070 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 70317072228d4a47877776b7fb4e21d1 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 87d4ebb338564421816a43cffeb42715 in current term 2: Already voted for candidate 70317072228d4a47877776b7fb4e21d1 in this term.
I20260809 22:53:16.149027 27966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:16.148582 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:42225 (local address 127.0.0.1:52786)
0809 22:53:16.148762 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0809 22:53:16.148860 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.113.190:42225: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97}
W20260809 22:53:16.149183 27031 leader_election.cc:334] T 00000000000000000000000000000000 P 70317072228d4a47877776b7fb4e21d1 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 460ec5412d65461a811d766224ae4ab6 (127.15.113.190:42225): Network error: Client connection negotiation failed: client connection to 127.15.113.190:42225: connect: Connection refused (error 111)
I20260809 22:53:16.149300 27031 leader_election.cc:302] T 00000000000000000000000000000000 P 70317072228d4a47877776b7fb4e21d1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70317072228d4a47877776b7fb4e21d1; no voters: 460ec5412d65461a811d766224ae4ab6, 87d4ebb338564421816a43cffeb42715
I20260809 22:53:16.149564 27961 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 70317072228d4a47877776b7fb4e21d1 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260809 22:53:16.149648 27965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:16.149270 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:42225 (local address 127.0.0.1:52790)
0809 22:53:16.149418 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0809 22:53:16.149530 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.113.190:42225: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260809 22:53:16.149813 26720 leader_election.cc:334] T 00000000000000000000000000000000 P 87d4ebb338564421816a43cffeb42715 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 460ec5412d65461a811d766224ae4ab6 (127.15.113.190:42225): Network error: Client connection negotiation failed: client connection to 127.15.113.190:42225: connect: Connection refused (error 111)
I20260809 22:53:16.149935 26720 leader_election.cc:302] T 00000000000000000000000000000000 P 87d4ebb338564421816a43cffeb42715 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 87d4ebb338564421816a43cffeb42715; no voters: 460ec5412d65461a811d766224ae4ab6, 70317072228d4a47877776b7fb4e21d1
I20260809 22:53:16.150162 27964 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 87d4ebb338564421816a43cffeb42715 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260809 22:53:16.609388 27974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:16.608573 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:42225 (local address 127.15.113.131:45415)
0809 22:53:16.609089 (+ 516us) negotiation.cc:107] Waiting for socket to connect
0809 22:53:16.609199 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.113.190:42225: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":404,"thread_start_us":310,"threads_started":1}
I20260809 22:53:16.610533 27975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:16.609859 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:42225 (local address 127.15.113.129:49579)
0809 22:53:16.610293 (+ 434us) negotiation.cc:107] Waiting for socket to connect
0809 22:53:16.610373 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.113.190:42225: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":333,"thread_start_us":236,"threads_started":1}
I20260809 22:53:16.615708 27977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:16.614975 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:42225 (local address 127.15.113.130:44527)
0809 22:53:16.615420 (+ 445us) negotiation.cc:107] Waiting for socket to connect
0809 22:53:16.615519 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.113.190:42225: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":337,"thread_start_us":206,"threads_started":1}
I20260809 22:53:17.611070 27978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:17.610281 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:42225 (local address 127.15.113.131:47691)
0809 22:53:17.610734 (+ 453us) negotiation.cc:107] Waiting for socket to connect
0809 22:53:17.610826 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.113.190:42225: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":337,"thread_start_us":287,"threads_started":1}
I20260809 22:53:17.612120 27980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:17.611354 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:42225 (local address 127.15.113.129:40653)
0809 22:53:17.611821 (+ 467us) negotiation.cc:107] Waiting for socket to connect
0809 22:53:17.611936 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.113.190:42225: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":357,"thread_start_us":239,"threads_started":1}
I20260809 22:53:17.617699 27981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:17.617002 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:42225 (local address 127.15.113.130:42349)
0809 22:53:17.617405 (+ 403us) negotiation.cc:107] Waiting for socket to connect
0809 22:53:17.617501 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.113.190:42225: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":314,"thread_start_us":221,"threads_started":1}
I20260809 22:53:17.719126 27982 raft_consensus.cc:482] T 00000000000000000000000000000000 P 87d4ebb338564421816a43cffeb42715 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 22:53:17.719339 27982 raft_consensus.cc:504] T 00000000000000000000000000000000 P 87d4ebb338564421816a43cffeb42715 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "460ec5412d65461a811d766224ae4ab6" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 42225 } } peers { permanent_uuid: "87d4ebb338564421816a43cffeb42715" member_type: VOTER last_known_addr { host: "127.15.113.189" port: 41115 } attrs { promote: false } } peers { permanent_uuid: "70317072228d4a47877776b7fb4e21d1" member_type: VOTER last_known_addr { host: "127.15.113.188" port: 43669 } attrs { promote: false } }
I20260809 22:53:17.719930 27982 leader_election.cc:288] T 00000000000000000000000000000000 P 87d4ebb338564421816a43cffeb42715 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 460ec5412d65461a811d766224ae4ab6 (127.15.113.190:42225), 70317072228d4a47877776b7fb4e21d1 (127.15.113.188:43669)
I20260809 22:53:17.721109 27983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:17.720136 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:42225 (local address 127.0.0.1:52996)
0809 22:53:17.720667 (+ 531us) negotiation.cc:107] Waiting for socket to connect
0809 22:53:17.720803 (+ 136us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.113.190:42225: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":363,"thread_start_us":224,"threads_started":1}
I20260809 22:53:17.721081 27070 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "87d4ebb338564421816a43cffeb42715" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "70317072228d4a47877776b7fb4e21d1" is_pre_election: true
I20260809 22:53:17.721346 27070 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 70317072228d4a47877776b7fb4e21d1 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 87d4ebb338564421816a43cffeb42715 in term 2.
I20260809 22:53:17.722035 26720 leader_election.cc:302] T 00000000000000000000000000000000 P 87d4ebb338564421816a43cffeb42715 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 70317072228d4a47877776b7fb4e21d1, 87d4ebb338564421816a43cffeb42715; no voters:
I20260809 22:53:17.722330 27982 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 87d4ebb338564421816a43cffeb42715 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260809 22:53:17.722434 27982 raft_consensus.cc:482] T 00000000000000000000000000000000 P 87d4ebb338564421816a43cffeb42715 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 22:53:17.722579 27982 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 87d4ebb338564421816a43cffeb42715 [term 2 FOLLOWER]: Advancing to term 3
I20260809 22:53:17.722790 27983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:17.722440 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:42225 (local address 127.0.0.1:53008)
0809 22:53:17.722579 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0809 22:53:17.722668 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.113.190:42225: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260809 22:53:17.722970 26720 leader_election.cc:334] T 00000000000000000000000000000000 P 87d4ebb338564421816a43cffeb42715 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 460ec5412d65461a811d766224ae4ab6 (127.15.113.190:42225): Network error: Client connection negotiation failed: client connection to 127.15.113.190:42225: connect: Connection refused (error 111)
I20260809 22:53:17.724673 27982 raft_consensus.cc:504] T 00000000000000000000000000000000 P 87d4ebb338564421816a43cffeb42715 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "460ec5412d65461a811d766224ae4ab6" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 42225 } } peers { permanent_uuid: "87d4ebb338564421816a43cffeb42715" member_type: VOTER last_known_addr { host: "127.15.113.189" port: 41115 } attrs { promote: false } } peers { permanent_uuid: "70317072228d4a47877776b7fb4e21d1" member_type: VOTER last_known_addr { host: "127.15.113.188" port: 43669 } attrs { promote: false } }
I20260809 22:53:17.725193 27982 leader_election.cc:288] T 00000000000000000000000000000000 P 87d4ebb338564421816a43cffeb42715 [CANDIDATE]: Term 3 election: Requested vote from peers 460ec5412d65461a811d766224ae4ab6 (127.15.113.190:42225), 70317072228d4a47877776b7fb4e21d1 (127.15.113.188:43669)
I20260809 22:53:17.725657 27983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:17.725277 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:42225 (local address 127.0.0.1:53016)
0809 22:53:17.725416 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0809 22:53:17.725523 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.113.190:42225: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260809 22:53:17.725792 27070 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "87d4ebb338564421816a43cffeb42715" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "70317072228d4a47877776b7fb4e21d1"
I20260809 22:53:17.725984 27070 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 70317072228d4a47877776b7fb4e21d1 [term 2 FOLLOWER]: Advancing to term 3
I20260809 22:53:17.726552 27983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:17.726253 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:42225 (local address 127.0.0.1:53030)
0809 22:53:17.726390 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0809 22:53:17.726455 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.113.190:42225: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260809 22:53:17.726716 26720 leader_election.cc:334] T 00000000000000000000000000000000 P 87d4ebb338564421816a43cffeb42715 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 460ec5412d65461a811d766224ae4ab6 (127.15.113.190:42225): Network error: Client connection negotiation failed: client connection to 127.15.113.190:42225: connect: Connection refused (error 111)
I20260809 22:53:17.728107 27070 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 70317072228d4a47877776b7fb4e21d1 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 87d4ebb338564421816a43cffeb42715 in term 3.
I20260809 22:53:17.728471 26720 leader_election.cc:302] T 00000000000000000000000000000000 P 87d4ebb338564421816a43cffeb42715 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 70317072228d4a47877776b7fb4e21d1, 87d4ebb338564421816a43cffeb42715; no voters: 460ec5412d65461a811d766224ae4ab6
I20260809 22:53:17.728730 27982 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 87d4ebb338564421816a43cffeb42715 [term 3 FOLLOWER]: Leader election won for term 3
I20260809 22:53:17.729110 27982 raft_consensus.cc:686] T 00000000000000000000000000000000 P 87d4ebb338564421816a43cffeb42715 [term 3 LEADER]: Becoming Leader. State: Replica: 87d4ebb338564421816a43cffeb42715, State: Running, Role: LEADER
I20260809 22:53:17.729504 27982 consensus_queue.cc:237] T 00000000000000000000000000000000 P 87d4ebb338564421816a43cffeb42715 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "460ec5412d65461a811d766224ae4ab6" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 42225 } } peers { permanent_uuid: "87d4ebb338564421816a43cffeb42715" member_type: VOTER last_known_addr { host: "127.15.113.189" port: 41115 } attrs { promote: false } } peers { permanent_uuid: "70317072228d4a47877776b7fb4e21d1" member_type: VOTER last_known_addr { host: "127.15.113.188" port: 43669 } attrs { promote: false } }
I20260809 22:53:17.731271 27986 sys_catalog.cc:455] T 00000000000000000000000000000000 P 87d4ebb338564421816a43cffeb42715 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 87d4ebb338564421816a43cffeb42715. Latest consensus state: current_term: 3 leader_uuid: "87d4ebb338564421816a43cffeb42715" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "460ec5412d65461a811d766224ae4ab6" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 42225 } } peers { permanent_uuid: "87d4ebb338564421816a43cffeb42715" member_type: VOTER last_known_addr { host: "127.15.113.189" port: 41115 } attrs { promote: false } } peers { permanent_uuid: "70317072228d4a47877776b7fb4e21d1" member_type: VOTER last_known_addr { host: "127.15.113.188" port: 43669 } attrs { promote: false } } }
I20260809 22:53:17.731513 27986 sys_catalog.cc:458] T 00000000000000000000000000000000 P 87d4ebb338564421816a43cffeb42715 [sys.catalog]: This master's current role is: LEADER
I20260809 22:53:17.732097 27988 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 22:53:17.735890 27988 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 22:53:17.736693 27988 catalog_manager.cc:1259] Loaded cluster ID: 2dcfd0b4c82745d78a39e01701586553
I20260809 22:53:17.736771 27988 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 22:53:17.737942 27988 catalog_manager.cc:1504] Loading token signing keys...
I20260809 22:53:17.738759 27988 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 87d4ebb338564421816a43cffeb42715: Loaded TSK: 0
I20260809 22:53:17.739612 27988 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 22:53:17.807564 26745 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50764:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 22:53:17.814683 26745 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 22:53:17.818652 27983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:17.818255 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:42225 (local address 127.0.0.1:53044)
0809 22:53:17.818407 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0809 22:53:17.818502 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.113.190:42225: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260809 22:53:17.818989 27070 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 70317072228d4a47877776b7fb4e21d1 [term 3 FOLLOWER]: Refusing update from remote peer 87d4ebb338564421816a43cffeb42715: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 3 index: 14. (index mismatch)
I20260809 22:53:17.819681 27983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:17.819294 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:42225 (local address 127.0.0.1:53060)
0809 22:53:17.819448 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0809 22:53:17.819533 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.113.190:42225: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260809 22:53:17.819680 27986 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 87d4ebb338564421816a43cffeb42715 [LEADER]: Connected to new peer: Peer: permanent_uuid: "70317072228d4a47877776b7fb4e21d1" member_type: VOTER last_known_addr { host: "127.15.113.188" port: 43669 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
W20260809 22:53:17.819943 26720 consensus_peers.cc:487] T 00000000000000000000000000000000 P 87d4ebb338564421816a43cffeb42715 -> Peer 460ec5412d65461a811d766224ae4ab6 (127.15.113.190:42225): Couldn't send request to peer 460ec5412d65461a811d766224ae4ab6. Status: Network error: Client connection negotiation failed: client connection to 127.15.113.190:42225: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260809 22:53:17.822630 27991 sys_catalog.cc:455] T 00000000000000000000000000000000 P 70317072228d4a47877776b7fb4e21d1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 87d4ebb338564421816a43cffeb42715. Latest consensus state: current_term: 3 leader_uuid: "87d4ebb338564421816a43cffeb42715" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "460ec5412d65461a811d766224ae4ab6" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 42225 } } peers { permanent_uuid: "87d4ebb338564421816a43cffeb42715" member_type: VOTER last_known_addr { host: "127.15.113.189" port: 41115 } attrs { promote: false } } peers { permanent_uuid: "70317072228d4a47877776b7fb4e21d1" member_type: VOTER last_known_addr { host: "127.15.113.188" port: 43669 } attrs { promote: false } } }
I20260809 22:53:17.822914 27991 sys_catalog.cc:458] T 00000000000000000000000000000000 P 70317072228d4a47877776b7fb4e21d1 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 22:53:17.823854 27986 sys_catalog.cc:455] T 00000000000000000000000000000000 P 87d4ebb338564421816a43cffeb42715 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "87d4ebb338564421816a43cffeb42715" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "460ec5412d65461a811d766224ae4ab6" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 42225 } } peers { permanent_uuid: "87d4ebb338564421816a43cffeb42715" member_type: VOTER last_known_addr { host: "127.15.113.189" port: 41115 } attrs { promote: false } } peers { permanent_uuid: "70317072228d4a47877776b7fb4e21d1" member_type: VOTER last_known_addr { host: "127.15.113.188" port: 43669 } attrs { promote: false } } }
I20260809 22:53:17.824155 27986 sys_catalog.cc:458] T 00000000000000000000000000000000 P 87d4ebb338564421816a43cffeb42715 [sys.catalog]: This master's current role is: LEADER
I20260809 22:53:17.824340 27991 sys_catalog.cc:455] T 00000000000000000000000000000000 P 70317072228d4a47877776b7fb4e21d1 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "87d4ebb338564421816a43cffeb42715" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "460ec5412d65461a811d766224ae4ab6" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 42225 } } peers { permanent_uuid: "87d4ebb338564421816a43cffeb42715" member_type: VOTER last_known_addr { host: "127.15.113.189" port: 41115 } attrs { promote: false } } peers { permanent_uuid: "70317072228d4a47877776b7fb4e21d1" member_type: VOTER last_known_addr { host: "127.15.113.188" port: 43669 } attrs { promote: false } } }
I20260809 22:53:17.824640 27991 sys_catalog.cc:458] T 00000000000000000000000000000000 P 70317072228d4a47877776b7fb4e21d1 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 22:53:17.854264 27983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:17.832468 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.129:38043 (local address 127.0.0.1:39814)
0809 22:53:17.832839 (+ 371us) negotiation.cc:107] Waiting for socket to connect
0809 22:53:17.832855 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 22:53:17.832999 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:53:17.848258 (+ 15259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:53:17.848262 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:53:17.848289 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:53:17.848624 (+ 335us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:53:17.848638 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:53:17.849938 (+ 1300us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:53:17.849943 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:53:17.851885 (+ 1942us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:53:17.851899 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:53:17.853454 (+ 1555us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:53:17.853460 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:53:17.853665 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:17.853700 (+ 35us) client_negotiation.cc:769] Sending connection context
0809 22:53:17.853800 (+ 100us) client_negotiation.cc:240] Negotiation successful
0809 22:53:17.853927 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":193,"mutex_wait_us":39}
I20260809 22:53:17.854314 27999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:17.832609 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39814 (local address 127.15.113.129:38043)
0809 22:53:17.847822 (+ 15213us) server_negotiation.cc:206] Beginning negotiation
0809 22:53:17.847828 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:53:17.847848 (+ 20us) server_negotiation.cc:407] Connection header received
0809 22:53:17.847920 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:53:17.847925 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:53:17.848009 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:53:17.848355 (+ 346us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:53:17.848820 (+ 465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:17.849817 (+ 997us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:17.852051 (+ 2234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:17.853257 (+ 1206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:17.853464 (+ 207us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:17.853539 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0809 22:53:17.853839 (+ 300us) server_negotiation.cc:299] Negotiation successful
0809 22:53:17.853975 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15077,"thread_start_us":15471,"threads_started":1}
I20260809 22:53:17.854959 28005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:17.833586 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.130:37171 (local address 127.0.0.1:47830)
0809 22:53:17.847698 (+ 14112us) negotiation.cc:107] Waiting for socket to connect
0809 22:53:17.847716 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 22:53:17.847840 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:53:17.848212 (+ 372us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:53:17.848217 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:53:17.848244 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:53:17.849118 (+ 874us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:53:17.849140 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:53:17.850454 (+ 1314us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:53:17.850459 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:53:17.852482 (+ 2023us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:53:17.852493 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:53:17.854085 (+ 1592us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:53:17.854094 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:53:17.854277 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:17.854355 (+ 78us) client_negotiation.cc:769] Sending connection context
0809 22:53:17.854441 (+ 86us) client_negotiation.cc:240] Negotiation successful
0809 22:53:17.854559 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14002,"mutex_wait_us":54,"thread_start_us":1573,"threads_started":1}
I20260809 22:53:17.855346 28003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:17.833295 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.131:32941 (local address 127.0.0.1:58128)
0809 22:53:17.834128 (+ 833us) negotiation.cc:107] Waiting for socket to connect
0809 22:53:17.834148 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 22:53:17.834271 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:53:17.848908 (+ 14637us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:53:17.848913 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:53:17.848938 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:53:17.849368 (+ 430us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:53:17.849380 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:53:17.850699 (+ 1319us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:53:17.850704 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:53:17.852519 (+ 1815us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:53:17.852534 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:53:17.854277 (+ 1743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:53:17.854284 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:53:17.854516 (+ 232us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:17.854548 (+ 32us) client_negotiation.cc:769] Sending connection context
0809 22:53:17.854650 (+ 102us) client_negotiation.cc:240] Negotiation successful
0809 22:53:17.854779 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":703,"mutex_wait_us":53,"thread_start_us":14620,"threads_started":1}
I20260809 22:53:17.855489 28001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:17.833405 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58128 (local address 127.15.113.131:32941)
0809 22:53:17.848456 (+ 15051us) server_negotiation.cc:206] Beginning negotiation
0809 22:53:17.848462 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:53:17.848480 (+ 18us) server_negotiation.cc:407] Connection header received
0809 22:53:17.848548 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:53:17.848553 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:53:17.848615 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:53:17.848702 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:53:17.849577 (+ 875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:17.850569 (+ 992us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:17.852710 (+ 2141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:17.853893 (+ 1183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:17.854033 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:17.854103 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0809 22:53:17.855038 (+ 935us) server_negotiation.cc:299] Negotiation successful
0809 22:53:17.855178 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14913,"thread_start_us":15558,"threads_started":1}
I20260809 22:53:17.855590 27236 tablet_service.cc:1467] Processing CreateTablet for tablet c255569d3137423e9a96b276e03dcf6d (DEFAULT_TABLE table=test-table [id=9e860368f53545a19c0204890f94b910]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:53:17.856014 28004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:17.833877 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47830 (local address 127.15.113.130:37171)
0809 22:53:17.842621 (+ 8744us) server_negotiation.cc:206] Beginning negotiation
0809 22:53:17.842627 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:53:17.847797 (+ 5170us) server_negotiation.cc:407] Connection header received
0809 22:53:17.847957 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:53:17.847962 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:53:17.848045 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:53:17.848779 (+ 734us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:53:17.849291 (+ 512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:17.850310 (+ 1019us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:17.852621 (+ 2311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:17.853736 (+ 1115us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:17.853882 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:17.853948 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0809 22:53:17.855568 (+ 1620us) server_negotiation.cc:299] Negotiation successful
0809 22:53:17.855707 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8612,"thread_start_us":8515,"threads_started":1}
I20260809 22:53:17.856046 27236 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c255569d3137423e9a96b276e03dcf6d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:53:17.856209 27715 tablet_service.cc:1467] Processing CreateTablet for tablet c255569d3137423e9a96b276e03dcf6d (DEFAULT_TABLE table=test-table [id=9e860368f53545a19c0204890f94b910]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:53:17.856629 27715 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c255569d3137423e9a96b276e03dcf6d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:53:17.856770 27489 tablet_service.cc:1467] Processing CreateTablet for tablet c255569d3137423e9a96b276e03dcf6d (DEFAULT_TABLE table=test-table [id=9e860368f53545a19c0204890f94b910]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:53:17.857200 27489 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c255569d3137423e9a96b276e03dcf6d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:53:17.861688 28007 tablet_bootstrap.cc:492] T c255569d3137423e9a96b276e03dcf6d P a0eed0b7d94e4c1f905abd17c6d0fd80: Bootstrap starting.
I20260809 22:53:17.863670 28007 tablet_bootstrap.cc:654] T c255569d3137423e9a96b276e03dcf6d P a0eed0b7d94e4c1f905abd17c6d0fd80: Neither blocks nor log segments found. Creating new log.
I20260809 22:53:17.863929 28009 tablet_bootstrap.cc:492] T c255569d3137423e9a96b276e03dcf6d P 200f1dd145df4993b427e20748be2114: Bootstrap starting.
I20260809 22:53:17.865562 28009 tablet_bootstrap.cc:654] T c255569d3137423e9a96b276e03dcf6d P 200f1dd145df4993b427e20748be2114: Neither blocks nor log segments found. Creating new log.
I20260809 22:53:17.866153 28007 tablet_bootstrap.cc:492] T c255569d3137423e9a96b276e03dcf6d P a0eed0b7d94e4c1f905abd17c6d0fd80: No bootstrap required, opened a new log
I20260809 22:53:17.866297 28007 ts_tablet_manager.cc:1397] T c255569d3137423e9a96b276e03dcf6d P a0eed0b7d94e4c1f905abd17c6d0fd80: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.001s
I20260809 22:53:17.867108 28007 raft_consensus.cc:348] T c255569d3137423e9a96b276e03dcf6d P a0eed0b7d94e4c1f905abd17c6d0fd80 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8e298532adf4449955b1b5dc49552c2" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 38043 } } peers { permanent_uuid: "a0eed0b7d94e4c1f905abd17c6d0fd80" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 32941 } } peers { permanent_uuid: "200f1dd145df4993b427e20748be2114" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 37171 } }
I20260809 22:53:17.867304 28007 raft_consensus.cc:374] T c255569d3137423e9a96b276e03dcf6d P a0eed0b7d94e4c1f905abd17c6d0fd80 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:53:17.867410 28007 raft_consensus.cc:729] T c255569d3137423e9a96b276e03dcf6d P a0eed0b7d94e4c1f905abd17c6d0fd80 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0eed0b7d94e4c1f905abd17c6d0fd80, State: Initialized, Role: FOLLOWER
I20260809 22:53:17.867848 28008 tablet_bootstrap.cc:492] T c255569d3137423e9a96b276e03dcf6d P b8e298532adf4449955b1b5dc49552c2: Bootstrap starting.
I20260809 22:53:17.867700 28007 consensus_queue.cc:260] T c255569d3137423e9a96b276e03dcf6d P a0eed0b7d94e4c1f905abd17c6d0fd80 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8e298532adf4449955b1b5dc49552c2" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 38043 } } peers { permanent_uuid: "a0eed0b7d94e4c1f905abd17c6d0fd80" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 32941 } } peers { permanent_uuid: "200f1dd145df4993b427e20748be2114" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 37171 } }
I20260809 22:53:17.868069 28009 tablet_bootstrap.cc:492] T c255569d3137423e9a96b276e03dcf6d P 200f1dd145df4993b427e20748be2114: No bootstrap required, opened a new log
I20260809 22:53:17.868266 28009 ts_tablet_manager.cc:1397] T c255569d3137423e9a96b276e03dcf6d P 200f1dd145df4993b427e20748be2114: Time spent bootstrapping tablet: real 0.004s user 0.004s sys 0.000s
I20260809 22:53:17.868543 28007 ts_tablet_manager.cc:1428] T c255569d3137423e9a96b276e03dcf6d P a0eed0b7d94e4c1f905abd17c6d0fd80: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260809 22:53:17.869285 28009 raft_consensus.cc:348] T c255569d3137423e9a96b276e03dcf6d P 200f1dd145df4993b427e20748be2114 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8e298532adf4449955b1b5dc49552c2" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 38043 } } peers { permanent_uuid: "a0eed0b7d94e4c1f905abd17c6d0fd80" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 32941 } } peers { permanent_uuid: "200f1dd145df4993b427e20748be2114" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 37171 } }
I20260809 22:53:17.869506 28009 raft_consensus.cc:374] T c255569d3137423e9a96b276e03dcf6d P 200f1dd145df4993b427e20748be2114 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:53:17.869553 28008 tablet_bootstrap.cc:654] T c255569d3137423e9a96b276e03dcf6d P b8e298532adf4449955b1b5dc49552c2: Neither blocks nor log segments found. Creating new log.
I20260809 22:53:17.869611 28009 raft_consensus.cc:729] T c255569d3137423e9a96b276e03dcf6d P 200f1dd145df4993b427e20748be2114 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 200f1dd145df4993b427e20748be2114, State: Initialized, Role: FOLLOWER
I20260809 22:53:17.870189 27978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:17.869232 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:42225 (local address 127.15.113.131:55985)
0809 22:53:17.869958 (+ 726us) negotiation.cc:107] Waiting for socket to connect
0809 22:53:17.870053 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.113.190:42225: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":596}
I20260809 22:53:17.872102 28008 tablet_bootstrap.cc:492] T c255569d3137423e9a96b276e03dcf6d P b8e298532adf4449955b1b5dc49552c2: No bootstrap required, opened a new log
I20260809 22:53:17.872205 27834 heartbeater.cc:503] Master 127.15.113.189:41115 was elected leader, sending a full tablet report...
I20260809 22:53:17.872290 28008 ts_tablet_manager.cc:1397] T c255569d3137423e9a96b276e03dcf6d P b8e298532adf4449955b1b5dc49552c2: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260809 22:53:17.872821 27978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:17.872369 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:42225 (local address 127.15.113.131:42261)
0809 22:53:17.872581 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0809 22:53:17.872663 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.113.190:42225: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120}
I20260809 22:53:17.873118 28008 raft_consensus.cc:348] T c255569d3137423e9a96b276e03dcf6d P b8e298532adf4449955b1b5dc49552c2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8e298532adf4449955b1b5dc49552c2" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 38043 } } peers { permanent_uuid: "a0eed0b7d94e4c1f905abd17c6d0fd80" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 32941 } } peers { permanent_uuid: "200f1dd145df4993b427e20748be2114" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 37171 } }
I20260809 22:53:17.873358 28008 raft_consensus.cc:374] T c255569d3137423e9a96b276e03dcf6d P b8e298532adf4449955b1b5dc49552c2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:53:17.873466 28008 raft_consensus.cc:729] T c255569d3137423e9a96b276e03dcf6d P b8e298532adf4449955b1b5dc49552c2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8e298532adf4449955b1b5dc49552c2, State: Initialized, Role: FOLLOWER
I20260809 22:53:17.873322 28009 consensus_queue.cc:260] T c255569d3137423e9a96b276e03dcf6d P 200f1dd145df4993b427e20748be2114 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8e298532adf4449955b1b5dc49552c2" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 38043 } } peers { permanent_uuid: "a0eed0b7d94e4c1f905abd17c6d0fd80" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 32941 } } peers { permanent_uuid: "200f1dd145df4993b427e20748be2114" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 37171 } }
I20260809 22:53:17.873688 28008 consensus_queue.cc:260] T c255569d3137423e9a96b276e03dcf6d P b8e298532adf4449955b1b5dc49552c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8e298532adf4449955b1b5dc49552c2" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 38043 } } peers { permanent_uuid: "a0eed0b7d94e4c1f905abd17c6d0fd80" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 32941 } } peers { permanent_uuid: "200f1dd145df4993b427e20748be2114" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 37171 } }
I20260809 22:53:17.874585 28008 ts_tablet_manager.cc:1428] T c255569d3137423e9a96b276e03dcf6d P b8e298532adf4449955b1b5dc49552c2: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260809 22:53:17.875391 27980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:17.874999 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:42225 (local address 127.15.113.129:35447)
0809 22:53:17.875133 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0809 22:53:17.875232 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.113.190:42225: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260809 22:53:17.876015 27981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:17.875644 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:42225 (local address 127.15.113.130:46653)
0809 22:53:17.875789 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0809 22:53:17.875882 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.113.190:42225: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260809 22:53:17.876737 27980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:17.876312 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:42225 (local address 127.15.113.129:55851)
0809 22:53:17.876516 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0809 22:53:17.876595 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.113.190:42225: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109}
I20260809 22:53:17.877103 27360 heartbeater.cc:503] Master 127.15.113.189:41115 was elected leader, sending a full tablet report...
I20260809 22:53:17.877825 28009 ts_tablet_manager.cc:1428] T c255569d3137423e9a96b276e03dcf6d P 200f1dd145df4993b427e20748be2114: Time spent starting tablet: real 0.009s user 0.002s sys 0.000s
I20260809 22:53:17.878762 27594 heartbeater.cc:503] Master 127.15.113.189:41115 was elected leader, sending a full tablet report...
I20260809 22:53:17.879432 27981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:17.879065 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:42225 (local address 127.15.113.130:43165)
0809 22:53:17.879223 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0809 22:53:17.879306 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.113.190:42225: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260809 22:53:18.055575 28017 raft_consensus.cc:482] T c255569d3137423e9a96b276e03dcf6d P b8e298532adf4449955b1b5dc49552c2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 22:53:18.055845 28017 raft_consensus.cc:504] T c255569d3137423e9a96b276e03dcf6d P b8e298532adf4449955b1b5dc49552c2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8e298532adf4449955b1b5dc49552c2" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 38043 } } peers { permanent_uuid: "a0eed0b7d94e4c1f905abd17c6d0fd80" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 32941 } } peers { permanent_uuid: "200f1dd145df4993b427e20748be2114" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 37171 } }
I20260809 22:53:18.057812 28017 leader_election.cc:288] T c255569d3137423e9a96b276e03dcf6d P b8e298532adf4449955b1b5dc49552c2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a0eed0b7d94e4c1f905abd17c6d0fd80 (127.15.113.131:32941), 200f1dd145df4993b427e20748be2114 (127.15.113.130:37171)
I20260809 22:53:18.065344 28004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:18.058220 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.113.129:39939 (local address 127.15.113.130:37171)
0809 22:53:18.058446 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0809 22:53:18.058453 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:53:18.059221 (+ 768us) server_negotiation.cc:407] Connection header received
0809 22:53:18.059363 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:53:18.059368 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:53:18.059442 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:53:18.059534 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:53:18.060132 (+ 598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:18.060941 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:18.063684 (+ 2743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:18.064441 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:18.064595 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:18.064692 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0809 22:53:18.064976 (+ 284us) server_negotiation.cc:299] Negotiation successful
0809 22:53:18.065108 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260809 22:53:18.065495 28018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:18.058398 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.130:37171 (local address 127.15.113.129:39939)
0809 22:53:18.059100 (+ 702us) negotiation.cc:107] Waiting for socket to connect
0809 22:53:18.059115 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 22:53:18.059247 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:53:18.059565 (+ 318us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:53:18.059568 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:53:18.059593 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:53:18.059977 (+ 384us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:53:18.059988 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:53:18.061062 (+ 1074us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:53:18.061068 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:53:18.063482 (+ 2414us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:53:18.063497 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:53:18.064633 (+ 1136us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:53:18.064640 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:53:18.064811 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:18.064835 (+ 24us) client_negotiation.cc:769] Sending connection context
0809 22:53:18.064949 (+ 114us) client_negotiation.cc:240] Negotiation successful
0809 22:53:18.065076 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":593,"mutex_wait_us":105,"thread_start_us":362,"threads_started":1}
I20260809 22:53:18.066259 27980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:18.058041 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.131:32941 (local address 127.15.113.129:56675)
0809 22:53:18.058636 (+ 595us) negotiation.cc:107] Waiting for socket to connect
0809 22:53:18.058662 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0809 22:53:18.058839 (+ 177us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:53:18.059411 (+ 572us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:53:18.059414 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:53:18.059436 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:53:18.059772 (+ 336us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:53:18.059786 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:53:18.061212 (+ 1426us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:53:18.061218 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:53:18.063480 (+ 2262us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:53:18.063497 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:53:18.065429 (+ 1932us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:53:18.065435 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:53:18.065767 (+ 332us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:18.065805 (+ 38us) client_negotiation.cc:769] Sending connection context
0809 22:53:18.065883 (+ 78us) client_negotiation.cc:240] Negotiation successful
0809 22:53:18.066010 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":470,"mutex_wait_us":44}
I20260809 22:53:18.066269 28001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:18.058408 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.113.129:56675 (local address 127.15.113.131:32941)
0809 22:53:18.058599 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0809 22:53:18.058604 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:53:18.058978 (+ 374us) server_negotiation.cc:407] Connection header received
0809 22:53:18.059063 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:53:18.059068 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:53:18.059148 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:53:18.059229 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:53:18.059926 (+ 697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:18.061073 (+ 1147us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:18.063742 (+ 2669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:18.065045 (+ 1303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:18.065198 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:18.065266 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0809 22:53:18.065926 (+ 660us) server_negotiation.cc:299] Negotiation successful
0809 22:53:18.066057 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260809 22:53:18.066367 27521 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c255569d3137423e9a96b276e03dcf6d" candidate_uuid: "b8e298532adf4449955b1b5dc49552c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "200f1dd145df4993b427e20748be2114" is_pre_election: true
I20260809 22:53:18.066635 27521 raft_consensus.cc:2454] T c255569d3137423e9a96b276e03dcf6d P 200f1dd145df4993b427e20748be2114 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b8e298532adf4449955b1b5dc49552c2 in term 0.
I20260809 22:53:18.067055 27751 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c255569d3137423e9a96b276e03dcf6d" candidate_uuid: "b8e298532adf4449955b1b5dc49552c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0eed0b7d94e4c1f905abd17c6d0fd80" is_pre_election: true
I20260809 22:53:18.067229 27163 leader_election.cc:302] T c255569d3137423e9a96b276e03dcf6d P b8e298532adf4449955b1b5dc49552c2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 200f1dd145df4993b427e20748be2114, b8e298532adf4449955b1b5dc49552c2; no voters:
I20260809 22:53:18.067458 27751 raft_consensus.cc:2454] T c255569d3137423e9a96b276e03dcf6d P a0eed0b7d94e4c1f905abd17c6d0fd80 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b8e298532adf4449955b1b5dc49552c2 in term 0.
I20260809 22:53:18.067657 28017 raft_consensus.cc:2781] T c255569d3137423e9a96b276e03dcf6d P b8e298532adf4449955b1b5dc49552c2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 22:53:18.067819 28017 raft_consensus.cc:482] T c255569d3137423e9a96b276e03dcf6d P b8e298532adf4449955b1b5dc49552c2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 22:53:18.067932 28017 raft_consensus.cc:3037] T c255569d3137423e9a96b276e03dcf6d P b8e298532adf4449955b1b5dc49552c2 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:53:18.070182 28017 raft_consensus.cc:504] T c255569d3137423e9a96b276e03dcf6d P b8e298532adf4449955b1b5dc49552c2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8e298532adf4449955b1b5dc49552c2" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 38043 } } peers { permanent_uuid: "a0eed0b7d94e4c1f905abd17c6d0fd80" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 32941 } } peers { permanent_uuid: "200f1dd145df4993b427e20748be2114" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 37171 } }
I20260809 22:53:18.070768 28017 leader_election.cc:288] T c255569d3137423e9a96b276e03dcf6d P b8e298532adf4449955b1b5dc49552c2 [CANDIDATE]: Term 1 election: Requested vote from peers a0eed0b7d94e4c1f905abd17c6d0fd80 (127.15.113.131:32941), 200f1dd145df4993b427e20748be2114 (127.15.113.130:37171)
I20260809 22:53:18.071117 27751 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c255569d3137423e9a96b276e03dcf6d" candidate_uuid: "b8e298532adf4449955b1b5dc49552c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0eed0b7d94e4c1f905abd17c6d0fd80"
I20260809 22:53:18.071267 27751 raft_consensus.cc:3037] T c255569d3137423e9a96b276e03dcf6d P a0eed0b7d94e4c1f905abd17c6d0fd80 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:53:18.071203 27521 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c255569d3137423e9a96b276e03dcf6d" candidate_uuid: "b8e298532adf4449955b1b5dc49552c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "200f1dd145df4993b427e20748be2114"
I20260809 22:53:18.071398 27521 raft_consensus.cc:3037] T c255569d3137423e9a96b276e03dcf6d P 200f1dd145df4993b427e20748be2114 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:53:18.073448 27751 raft_consensus.cc:2454] T c255569d3137423e9a96b276e03dcf6d P a0eed0b7d94e4c1f905abd17c6d0fd80 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8e298532adf4449955b1b5dc49552c2 in term 1.
I20260809 22:53:18.073464 27521 raft_consensus.cc:2454] T c255569d3137423e9a96b276e03dcf6d P 200f1dd145df4993b427e20748be2114 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8e298532adf4449955b1b5dc49552c2 in term 1.
I20260809 22:53:18.073810 27163 leader_election.cc:302] T c255569d3137423e9a96b276e03dcf6d P b8e298532adf4449955b1b5dc49552c2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 200f1dd145df4993b427e20748be2114, b8e298532adf4449955b1b5dc49552c2; no voters:
I20260809 22:53:18.074064 28017 raft_consensus.cc:2781] T c255569d3137423e9a96b276e03dcf6d P b8e298532adf4449955b1b5dc49552c2 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 22:53:18.074508 28017 raft_consensus.cc:686] T c255569d3137423e9a96b276e03dcf6d P b8e298532adf4449955b1b5dc49552c2 [term 1 LEADER]: Becoming Leader. State: Replica: b8e298532adf4449955b1b5dc49552c2, State: Running, Role: LEADER
I20260809 22:53:18.075089 28017 consensus_queue.cc:237] T c255569d3137423e9a96b276e03dcf6d P b8e298532adf4449955b1b5dc49552c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8e298532adf4449955b1b5dc49552c2" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 38043 } } peers { permanent_uuid: "a0eed0b7d94e4c1f905abd17c6d0fd80" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 32941 } } peers { permanent_uuid: "200f1dd145df4993b427e20748be2114" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 37171 } }
I20260809 22:53:18.075475 27980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:18.075038 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:42225 (local address 127.15.113.129:39079)
0809 22:53:18.075227 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0809 22:53:18.075324 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.113.190:42225: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260809 22:53:18.078593 26748 catalog_manager.cc:5495] T c255569d3137423e9a96b276e03dcf6d P b8e298532adf4449955b1b5dc49552c2 reported cstate change: term changed from 0 to 1, leader changed from <none> to b8e298532adf4449955b1b5dc49552c2 (127.15.113.129). New cstate: current_term: 1 leader_uuid: "b8e298532adf4449955b1b5dc49552c2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8e298532adf4449955b1b5dc49552c2" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 38043 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a0eed0b7d94e4c1f905abd17c6d0fd80" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 32941 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "200f1dd145df4993b427e20748be2114" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 37171 } health_report { overall_health: UNKNOWN } } }
I20260809 22:53:18.136137 27999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:18.131169 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39816 (local address 127.15.113.129:38043)
0809 22:53:18.131350 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0809 22:53:18.131356 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:53:18.131375 (+ 19us) server_negotiation.cc:407] Connection header received
0809 22:53:18.131570 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:53:18.131574 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:53:18.131664 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:53:18.131795 (+ 131us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 22:53:18.132149 (+ 354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:18.132855 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:18.134137 (+ 1282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:18.134563 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:18.134690 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:18.135055 (+ 365us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 22:53:18.135578 (+ 523us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 22:53:18.135731 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0809 22:53:18.135807 (+ 76us) server_negotiation.cc:299] Negotiation successful
0809 22:53:18.135886 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260809 22:53:18.140405 27521 raft_consensus.cc:1261] T c255569d3137423e9a96b276e03dcf6d P 200f1dd145df4993b427e20748be2114 [term 1 FOLLOWER]: Refusing update from remote peer b8e298532adf4449955b1b5dc49552c2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 22:53:18.140726 27751 raft_consensus.cc:1261] T c255569d3137423e9a96b276e03dcf6d P a0eed0b7d94e4c1f905abd17c6d0fd80 [term 1 FOLLOWER]: Refusing update from remote peer b8e298532adf4449955b1b5dc49552c2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 22:53:18.141176 28017 consensus_queue.cc:1035] T c255569d3137423e9a96b276e03dcf6d P b8e298532adf4449955b1b5dc49552c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "200f1dd145df4993b427e20748be2114" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 37171 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 22:53:18.141757 28017 consensus_queue.cc:1035] T c255569d3137423e9a96b276e03dcf6d P b8e298532adf4449955b1b5dc49552c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0eed0b7d94e4c1f905abd17c6d0fd80" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 32941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 22:53:18.143384 27981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:18.142939 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:42225 (local address 127.15.113.130:48411)
0809 22:53:18.143108 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0809 22:53:18.143229 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.113.190:42225: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260809 22:53:18.144562 27978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:18.144184 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:42225 (local address 127.15.113.131:40203)
0809 22:53:18.144336 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0809 22:53:18.144413 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.113.190:42225: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260809 22:53:18.149830 27978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:18.149179 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:42225 (local address 127.15.113.131:55199)
0809 22:53:18.149528 (+ 349us) negotiation.cc:107] Waiting for socket to connect
0809 22:53:18.149627 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.113.190:42225: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":224}
I20260809 22:53:18.151366 28034 mvcc.cc:205] Tried to move back new op lower bound from 7316750328375312384 to 7316750328119549952. Current Snapshot: MvccSnapshot[applied={T|T < 7316750328375312384}]
I20260809 22:53:18.153285 27981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:18.152488 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:42225 (local address 127.15.113.130:33005)
0809 22:53:18.152992 (+ 504us) negotiation.cc:107] Waiting for socket to connect
0809 22:53:18.153087 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.113.190:42225: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":361}
I20260809 22:53:18.157230 27980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:18.156800 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:42225 (local address 127.15.113.129:52077)
0809 22:53:18.156979 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0809 22:53:18.157068 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.113.190:42225: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260809 22:53:18.158311 27980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:18.157878 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:42225 (local address 127.15.113.129:46375)
0809 22:53:18.158043 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0809 22:53:18.158129 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.113.190:42225: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260809 22:53:18.164810 26744 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50764:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 22:53:18.165110 26744 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50764:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 22:53:18.170398 26744 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 87d4ebb338564421816a43cffeb42715: Sending DeleteTablet for 3 replicas of tablet c255569d3137423e9a96b276e03dcf6d
I20260809 22:53:18.171409 27715 tablet_service.cc:1514] Processing DeleteTablet for tablet c255569d3137423e9a96b276e03dcf6d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:53:18 UTC) from {username='oryx', principal='oryx/127.15.113.189@KRBTEST.COM'} at 127.0.0.1:58128
I20260809 22:53:18.171684 27489 tablet_service.cc:1514] Processing DeleteTablet for tablet c255569d3137423e9a96b276e03dcf6d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:53:18 UTC) from {username='oryx', principal='oryx/127.15.113.189@KRBTEST.COM'} at 127.0.0.1:47830
I20260809 22:53:18.172094 28049 tablet_replica.cc:330] T c255569d3137423e9a96b276e03dcf6d P a0eed0b7d94e4c1f905abd17c6d0fd80: stopping tablet replica
I20260809 22:53:18.172274 28049 raft_consensus.cc:2229] T c255569d3137423e9a96b276e03dcf6d P a0eed0b7d94e4c1f905abd17c6d0fd80 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 22:53:18.172580 27236 tablet_service.cc:1514] Processing DeleteTablet for tablet c255569d3137423e9a96b276e03dcf6d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:53:18 UTC) from {username='oryx', principal='oryx/127.15.113.189@KRBTEST.COM'} at 127.0.0.1:39814
I20260809 22:53:18.172660 28049 raft_consensus.cc:2258] T c255569d3137423e9a96b276e03dcf6d P a0eed0b7d94e4c1f905abd17c6d0fd80 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 22:53:18.173296 15814 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskItYPJR/build/asan/bin/kudu pid 27007
I20260809 22:53:18.176518 28049 ts_tablet_manager.cc:1905] T c255569d3137423e9a96b276e03dcf6d P a0eed0b7d94e4c1f905abd17c6d0fd80: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 22:53:18.183116 28049 ts_tablet_manager.cc:1918] T c255569d3137423e9a96b276e03dcf6d P a0eed0b7d94e4c1f905abd17c6d0fd80: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 22:53:18.183306 28049 log.cc:1196] T c255569d3137423e9a96b276e03dcf6d P a0eed0b7d94e4c1f905abd17c6d0fd80: Deleting WAL directory at /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315935617919-15814-0/minicluster-data/ts-2/wal/wals/c255569d3137423e9a96b276e03dcf6d
I20260809 22:53:18.183873 28049 ts_tablet_manager.cc:1939] T c255569d3137423e9a96b276e03dcf6d P a0eed0b7d94e4c1f905abd17c6d0fd80: Deleting consensus metadata
I20260809 22:53:18.185060 26721 catalog_manager.cc:4841] TS a0eed0b7d94e4c1f905abd17c6d0fd80 (127.15.113.131:32941): tablet c255569d3137423e9a96b276e03dcf6d (table test-table [id=9e860368f53545a19c0204890f94b910]) successfully deleted
I20260809 22:53:18.190073 28050 tablet_replica.cc:330] T c255569d3137423e9a96b276e03dcf6d P 200f1dd145df4993b427e20748be2114: stopping tablet replica
I20260809 22:53:18.190264 28050 raft_consensus.cc:2229] T c255569d3137423e9a96b276e03dcf6d P 200f1dd145df4993b427e20748be2114 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 22:53:18.190567 28050 raft_consensus.cc:2258] T c255569d3137423e9a96b276e03dcf6d P 200f1dd145df4993b427e20748be2114 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 22:53:18.190982 28051 tablet_replica.cc:330] T c255569d3137423e9a96b276e03dcf6d P b8e298532adf4449955b1b5dc49552c2: stopping tablet replica
I20260809 22:53:18.192626 28051 raft_consensus.cc:2229] T c255569d3137423e9a96b276e03dcf6d P b8e298532adf4449955b1b5dc49552c2 [term 1 LEADER]: Raft consensus shutting down.
I20260809 22:53:18.193346 28051 raft_consensus.cc:2258] T c255569d3137423e9a96b276e03dcf6d P b8e298532adf4449955b1b5dc49552c2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 22:53:18.193797 28050 ts_tablet_manager.cc:1905] T c255569d3137423e9a96b276e03dcf6d P 200f1dd145df4993b427e20748be2114: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 22:53:18.195250 28051 ts_tablet_manager.cc:1905] T c255569d3137423e9a96b276e03dcf6d P b8e298532adf4449955b1b5dc49552c2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 22:53:18.199457 28050 ts_tablet_manager.cc:1918] T c255569d3137423e9a96b276e03dcf6d P 200f1dd145df4993b427e20748be2114: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 22:53:18.199575 28050 log.cc:1196] T c255569d3137423e9a96b276e03dcf6d P 200f1dd145df4993b427e20748be2114: Deleting WAL directory at /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315935617919-15814-0/minicluster-data/ts-1/wal/wals/c255569d3137423e9a96b276e03dcf6d
I20260809 22:53:18.200037 28050 ts_tablet_manager.cc:1939] T c255569d3137423e9a96b276e03dcf6d P 200f1dd145df4993b427e20748be2114: Deleting consensus metadata
I20260809 22:53:18.200979 26723 catalog_manager.cc:4841] TS 200f1dd145df4993b427e20748be2114 (127.15.113.130:37171): tablet c255569d3137423e9a96b276e03dcf6d (table test-table [id=9e860368f53545a19c0204890f94b910]) successfully deleted
I20260809 22:53:18.204766 27999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:18.174093 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39822 (local address 127.15.113.129:38043)
0809 22:53:18.174256 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0809 22:53:18.174262 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:53:18.189166 (+ 14904us) server_negotiation.cc:407] Connection header received
0809 22:53:18.189363 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:53:18.189368 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:53:18.189487 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:53:18.189605 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:53:18.191115 (+ 1510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:18.192355 (+ 1240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:18.193902 (+ 1547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:18.194537 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:18.194672 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:18.197832 (+ 3160us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:53:18.197846 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:53:18.197866 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:53:18.197903 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:53:18.201861 (+ 3958us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:53:18.202536 (+ 675us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:53:18.202546 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:53:18.202549 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:53:18.202735 (+ 186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:53:18.203099 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:53:18.203104 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:53:18.203108 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:53:18.203477 (+ 369us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:53:18.203812 (+ 335us) server_negotiation.cc:1035] Waiting for connection context
0809 22:53:18.204156 (+ 344us) server_negotiation.cc:299] Negotiation successful
0809 22:53:18.204428 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260809 22:53:18.205401 27338 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:53:18.261723 28003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:18.261295 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:42225 (local address 127.0.0.1:53074)
0809 22:53:18.261468 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0809 22:53:18.261563 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.113.190:42225: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260809 22:53:18.262738 28003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:18.262381 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:42225 (local address 127.0.0.1:53084)
0809 22:53:18.262525 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0809 22:53:18.262605 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.113.190:42225: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260809 22:53:18.308848 28051 ts_tablet_manager.cc:1918] T c255569d3137423e9a96b276e03dcf6d P b8e298532adf4449955b1b5dc49552c2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 22:53:18.309120 28051 log.cc:1196] T c255569d3137423e9a96b276e03dcf6d P b8e298532adf4449955b1b5dc49552c2: Deleting WAL directory at /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315935617919-15814-0/minicluster-data/ts-0/wal/wals/c255569d3137423e9a96b276e03dcf6d
I20260809 22:53:18.309600 15814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskItYPJR/build/asan/bin/kudu with pid 27007
I20260809 22:53:18.309721 28051 ts_tablet_manager.cc:1939] T c255569d3137423e9a96b276e03dcf6d P b8e298532adf4449955b1b5dc49552c2: Deleting consensus metadata
I20260809 22:53:18.334563 15814 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskItYPJR/build/asan/bin/kudu pid 27374
I20260809 22:53:18.335376 28003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:18.334902 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.129:38043 (local address 127.0.0.1:39830)
0809 22:53:18.335098 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0809 22:53:18.335209 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.113.129:38043: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
W20260809 22:53:18.335644 26723 catalog_manager.cc:4584] TS b8e298532adf4449955b1b5dc49552c2 (127.15.113.129:38043): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet c255569d3137423e9a96b276e03dcf6d: Network error: Client connection negotiation failed: client connection to 127.15.113.129:38043: connect: Connection refused (error 111)
I20260809 22:53:18.347714 28004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:18.335328 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47846 (local address 127.15.113.130:37171)
0809 22:53:18.335502 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0809 22:53:18.335508 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:53:18.335590 (+ 82us) server_negotiation.cc:407] Connection header received
0809 22:53:18.335790 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:53:18.335796 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:53:18.335889 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:53:18.335980 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:53:18.337355 (+ 1375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:18.338089 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:18.339323 (+ 1234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:18.339742 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:18.339856 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:18.342537 (+ 2681us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:53:18.342552 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:53:18.342571 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:53:18.342603 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:53:18.345109 (+ 2506us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:53:18.345756 (+ 647us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:53:18.345764 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:53:18.345766 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:53:18.345904 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:53:18.346266 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:53:18.346271 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:53:18.346274 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:53:18.346620 (+ 346us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:53:18.346924 (+ 304us) server_negotiation.cc:1035] Waiting for connection context
0809 22:53:18.347211 (+ 287us) server_negotiation.cc:299] Negotiation successful
0809 22:53:18.347418 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260809 22:53:18.348325 27579 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:53:18.387955 28003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:18.387437 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.129:38043 (local address 127.0.0.1:39832)
0809 22:53:18.387662 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0809 22:53:18.387770 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.113.129:38043: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
I20260809 22:53:18.389003 28003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:18.388577 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.129:38043 (local address 127.0.0.1:39834)
0809 22:53:18.388748 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0809 22:53:18.388837 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.113.129:38043: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260809 22:53:18.440104 15814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskItYPJR/build/asan/bin/kudu with pid 27374
I20260809 22:53:18.458690 28003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:18.458175 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.129:38043 (local address 127.0.0.1:39848)
0809 22:53:18.458377 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0809 22:53:18.458498 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.113.129:38043: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260809 22:53:18.459631 28003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:18.459295 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.129:38043 (local address 127.0.0.1:39860)
0809 22:53:18.459440 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0809 22:53:18.459513 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.113.129:38043: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260809 22:53:18.464519 15814 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskItYPJR/build/asan/bin/kudu pid 27618
I20260809 22:53:18.477449 28001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:18.465255 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58132 (local address 127.15.113.131:32941)
0809 22:53:18.465441 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0809 22:53:18.465448 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:53:18.465467 (+ 19us) server_negotiation.cc:407] Connection header received
0809 22:53:18.465620 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:53:18.465626 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:53:18.465702 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:53:18.465799 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:53:18.467153 (+ 1354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:18.467907 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:18.469280 (+ 1373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:18.469704 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:18.469814 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:18.472251 (+ 2437us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:53:18.472266 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:53:18.472283 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:53:18.472314 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:53:18.474855 (+ 2541us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:53:18.475472 (+ 617us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:53:18.475482 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:53:18.475485 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:53:18.475612 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:53:18.475969 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:53:18.475974 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:53:18.475978 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:53:18.476320 (+ 342us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:53:18.476636 (+ 316us) server_negotiation.cc:1035] Waiting for connection context
0809 22:53:18.476926 (+ 290us) server_negotiation.cc:299] Negotiation successful
0809 22:53:18.477150 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260809 22:53:18.478085 27824 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:53:18.567588 28003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:18.567029 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.129:38043 (local address 127.0.0.1:39874)
0809 22:53:18.567264 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0809 22:53:18.567390 (+ 126us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.113.129:38043: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":88}
I20260809 22:53:18.568611 28003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:18.568271 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.129:38043 (local address 127.0.0.1:39884)
0809 22:53:18.568407 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0809 22:53:18.568487 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.113.129:38043: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260809 22:53:18.595160 15814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskItYPJR/build/asan/bin/kudu with pid 27618
I20260809 22:53:18.624778 15814 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskItYPJR/build/asan/bin/kudu pid 26529
I20260809 22:53:18.625856 26794 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:53:18.747726 28003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:18.747063 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.129:38043 (local address 127.0.0.1:35862)
0809 22:53:18.747395 (+ 332us) negotiation.cc:107] Waiting for socket to connect
0809 22:53:18.747500 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.113.129:38043: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":194}
I20260809 22:53:18.747865 15814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskItYPJR/build/asan/bin/kudu with pid 26529
I20260809 22:53:18.777748 15814 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskItYPJR/build/asan/bin/kudu pid 26693
I20260809 22:53:18.778795 27100 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:53:18.894807 15814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskItYPJR/build/asan/bin/kudu with pid 26693
2026-08-09T22:53:18Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (11945 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315935617919-15814-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 22:53:19 dist-test-slave-ncp9 krb5kdc[28066](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 22:53:19 dist-test-slave-ncp9 krb5kdc[28066](info): set up 2 sockets
Aug 09 22:53:19 dist-test-slave-ncp9 krb5kdc[28066](info): commencing operation
krb5kdc: starting...
W20260809 22:53:20.126690 15814 mini_kdc.cc:121] Time spent starting KDC: real 1.176s user 0.000s sys 0.032s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315935617919-15814-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315935617919-15814-0/krb5kdc/test-user.keytab.
Aug 09 22:53:20 dist-test-slave-ncp9 krb5kdc[28066](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316000, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T22:53:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T22:53:20Z Disabled control of system clock
WARNING: no policy specified for oryx/127.15.113.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.15.113.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.15.113.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315935617919-15814-0/krb5kdc/oryx_127.15.113.190.keytab.
Entry for principal oryx/127.15.113.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315935617919-15814-0/krb5kdc/oryx_127.15.113.190.keytab.
I20260809 22:53:20.350262 15814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskItYPJR/build/asan/bin/kudu
/tmp/dist-test-taskItYPJR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315935617919-15814-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315935617919-15814-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315935617919-15814-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315935617919-15814-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.113.190:34881
--webserver_interface=127.15.113.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315935617919-15814-0/krb5kdc/oryx_127.15.113.190.keytab
--principal=oryx/127.15.113.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.113.148:33227
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.113.190:34881
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315935617919-15814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315935617919-15814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315935617919-15814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:53:20.669162 28082 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:53:20.669430 28082 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:53:20.669561 28082 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:53:20.676539 28082 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 22:53:20.676658 28082 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260809 22:53:20.676729 28082 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:53:20.676811 28082 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:53:20.676877 28082 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 22:53:20.676980 28082 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 22:53:20.686474 28082 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.113.148:33227
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315935617919-15814-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315935617919-15814-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.113.190:34881
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315935617919-15814-0/krb5kdc/oryx_127.15.113.190.keytab
--principal=oryx/127.15.113.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.113.190:34881
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315935617919-15814-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.113.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315935617919-15814-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:53:20.687696 28082 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:53:20.689049 28082 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:53:20.700634 28089 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:53:20.700557 28092 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:53:20.700677 28090 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:53:20.701296 28082 server_base.cc:1034] running on GCE node
I20260809 22:53:20.702045 28082 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:53:20.703406 28082 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:53:20.704631 28082 hybrid_clock.cc:648] HybridClock initialized: now 1786316000704556 us; error 41 us; skew 500 ppm
I20260809 22:53:20.704978 28082 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:53:20 dist-test-slave-ncp9 krb5kdc[28066](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316000, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.113.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:53:20.707949 28082 init.cc:377] Logged in from keytab as oryx/127.15.113.190@KRBTEST.COM (short username oryx)
I20260809 22:53:20.709465 28082 webserver.cc:460] Webserver started at http://127.15.113.190:40877/ using document root <none> and password file <none>
I20260809 22:53:20.710158 28082 fs_manager.cc:359] Metadata directory not provided
I20260809 22:53:20.710263 28082 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:53:20.710577 28082 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:53:20.713630 28082 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315935617919-15814-0/minicluster-data/master-0/data/instance:
uuid: "f37bab48ada649e6bd170b082f14874f"
format_stamp: "Formatted at 2026-08-09 22:53:20 on dist-test-slave-ncp9"
server_key: "1e219d7fa0adf3a16d530497f22da41d"
server_key_iv: "ec9728c8d73850b1d6eadaa064a471f7"
server_key_version: "encryptionkey@0"
I20260809 22:53:20.714427 28082 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315935617919-15814-0/minicluster-data/master-0/wal/instance:
uuid: "f37bab48ada649e6bd170b082f14874f"
format_stamp: "Formatted at 2026-08-09 22:53:20 on dist-test-slave-ncp9"
server_key: "1e219d7fa0adf3a16d530497f22da41d"
server_key_iv: "ec9728c8d73850b1d6eadaa064a471f7"
server_key_version: "encryptionkey@0"
I20260809 22:53:20.719875 28082 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260809 22:53:20.723346 28101 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:53:20.724663 28082 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.001s
I20260809 22:53:20.724819 28082 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315935617919-15814-0/minicluster-data/master-0/data,/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315935617919-15814-0/minicluster-data/master-0/wal
uuid: "f37bab48ada649e6bd170b082f14874f"
format_stamp: "Formatted at 2026-08-09 22:53:20 on dist-test-slave-ncp9"
server_key: "1e219d7fa0adf3a16d530497f22da41d"
server_key_iv: "ec9728c8d73850b1d6eadaa064a471f7"
server_key_version: "encryptionkey@0"
I20260809 22:53:20.725085 28082 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315935617919-15814-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315935617919-15814-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315935617919-15814-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:53:20.744359 28082 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:53:20.748185 28082 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:53:20.748431 28082 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:53:20.768672 28082 rpc_server.cc:307] RPC server started. Bound to: 127.15.113.190:34881
I20260809 22:53:20.768730 28196 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.113.190:34881 every 8 connection(s)
I20260809 22:53:20.769954 28082 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315935617919-15814-0/minicluster-data/master-0/data/info.pb
I20260809 22:53:20.773514 28198 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:53:20.779211 15814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskItYPJR/build/asan/bin/kudu as pid 28082
I20260809 22:53:20.779433 15814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315935617919-15814-0/minicluster-data/master-0/wal/instance
I20260809 22:53:20.780297 15814 external_mini_cluster.cc:1428] Setting key 340bb7558a87d98b47792ebdd8078e37
I20260809 22:53:20.784539 28198 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f37bab48ada649e6bd170b082f14874f: Bootstrap starting.
I20260809 22:53:20.788038 28198 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f37bab48ada649e6bd170b082f14874f: Neither blocks nor log segments found. Creating new log.
I20260809 22:53:20.789161 28198 log.cc:824] T 00000000000000000000000000000000 P f37bab48ada649e6bd170b082f14874f: Log is configured to *not* fsync() on all Append() calls
I20260809 22:53:20.791987 28198 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f37bab48ada649e6bd170b082f14874f: No bootstrap required, opened a new log
Aug 09 22:53:20 dist-test-slave-ncp9 krb5kdc[28066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316000, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.15.113.190@KRBTEST.COM
I20260809 22:53:20.799705 28198 raft_consensus.cc:348] T 00000000000000000000000000000000 P f37bab48ada649e6bd170b082f14874f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f37bab48ada649e6bd170b082f14874f" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 34881 } }
I20260809 22:53:20.800043 28198 raft_consensus.cc:374] T 00000000000000000000000000000000 P f37bab48ada649e6bd170b082f14874f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:53:20.800143 28198 raft_consensus.cc:729] T 00000000000000000000000000000000 P f37bab48ada649e6bd170b082f14874f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f37bab48ada649e6bd170b082f14874f, State: Initialized, Role: FOLLOWER
I20260809 22:53:20.800756 28198 consensus_queue.cc:260] T 00000000000000000000000000000000 P f37bab48ada649e6bd170b082f14874f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f37bab48ada649e6bd170b082f14874f" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 34881 } }
I20260809 22:53:20.801025 28198 raft_consensus.cc:388] T 00000000000000000000000000000000 P f37bab48ada649e6bd170b082f14874f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 22:53:20.801242 28198 raft_consensus.cc:482] T 00000000000000000000000000000000 P f37bab48ada649e6bd170b082f14874f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 22:53:20.801415 28198 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f37bab48ada649e6bd170b082f14874f [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:53:20.802302 28202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:20.782413 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49110 (local address 127.15.113.190:34881)
0809 22:53:20.783172 (+ 759us) server_negotiation.cc:206] Beginning negotiation
0809 22:53:20.783183 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 22:53:20.783223 (+ 40us) server_negotiation.cc:407] Connection header received
0809 22:53:20.784919 (+ 1696us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:53:20.784947 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:53:20.785316 (+ 369us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:53:20.785611 (+ 295us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:53:20.787358 (+ 1747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:20.788680 (+ 1322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:20.790266 (+ 1586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:20.790918 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:20.791042 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:20.795428 (+ 4386us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:53:20.795436 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:53:20.795449 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:53:20.795498 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:53:20.798390 (+ 2892us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:53:20.799017 (+ 627us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:53:20.799023 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:53:20.799032 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:53:20.799190 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:53:20.799568 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:53:20.799571 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:53:20.799574 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:53:20.800115 (+ 541us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:53:20.800398 (+ 283us) server_negotiation.cc:1035] Waiting for connection context
0809 22:53:20.800837 (+ 439us) server_negotiation.cc:299] Negotiation successful
0809 22:53:20.801269 (+ 432us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":563,"thread_start_us":363,"threads_started":1}
I20260809 22:53:20.804114 28198 raft_consensus.cc:504] T 00000000000000000000000000000000 P f37bab48ada649e6bd170b082f14874f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f37bab48ada649e6bd170b082f14874f" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 34881 } }
I20260809 22:53:20.804590 28198 leader_election.cc:302] T 00000000000000000000000000000000 P f37bab48ada649e6bd170b082f14874f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f37bab48ada649e6bd170b082f14874f; no voters:
I20260809 22:53:20.805226 28198 leader_election.cc:288] T 00000000000000000000000000000000 P f37bab48ada649e6bd170b082f14874f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 22:53:20.805325 28205 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f37bab48ada649e6bd170b082f14874f [term 1 FOLLOWER]: Leader election won for term 1
I20260809 22:53:20.805995 28205 raft_consensus.cc:686] T 00000000000000000000000000000000 P f37bab48ada649e6bd170b082f14874f [term 1 LEADER]: Becoming Leader. State: Replica: f37bab48ada649e6bd170b082f14874f, State: Running, Role: LEADER
I20260809 22:53:20.806516 28205 consensus_queue.cc:237] T 00000000000000000000000000000000 P f37bab48ada649e6bd170b082f14874f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f37bab48ada649e6bd170b082f14874f" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 34881 } }
I20260809 22:53:20.807278 28198 sys_catalog.cc:564] T 00000000000000000000000000000000 P f37bab48ada649e6bd170b082f14874f [sys.catalog]: configured and running, proceeding with master startup.
I20260809 22:53:20.809267 28208 sys_catalog.cc:455] T 00000000000000000000000000000000 P f37bab48ada649e6bd170b082f14874f [sys.catalog]: SysCatalogTable state changed. Reason: New leader f37bab48ada649e6bd170b082f14874f. Latest consensus state: current_term: 1 leader_uuid: "f37bab48ada649e6bd170b082f14874f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f37bab48ada649e6bd170b082f14874f" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 34881 } } }
I20260809 22:53:20.809798 28208 sys_catalog.cc:458] T 00000000000000000000000000000000 P f37bab48ada649e6bd170b082f14874f [sys.catalog]: This master's current role is: LEADER
I20260809 22:53:20.810415 28206 sys_catalog.cc:455] T 00000000000000000000000000000000 P f37bab48ada649e6bd170b082f14874f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f37bab48ada649e6bd170b082f14874f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f37bab48ada649e6bd170b082f14874f" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 34881 } } }
I20260809 22:53:20.812017 28206 sys_catalog.cc:458] T 00000000000000000000000000000000 P f37bab48ada649e6bd170b082f14874f [sys.catalog]: This master's current role is: LEADER
I20260809 22:53:20.813364 28216 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 22:53:20.818496 28216 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 22:53:20.827632 28216 catalog_manager.cc:1347] Generated new cluster ID: 406c955e738d4ff9b564c3159216615e
I20260809 22:53:20.827739 28216 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 22:53:20.841572 28216 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 22:53:20.842828 28216 catalog_manager.cc:1504] Loading token signing keys...
I20260809 22:53:20.851410 28216 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f37bab48ada649e6bd170b082f14874f: Generated new TSK 0
I20260809 22:53:20.852185 28216 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.15.113.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.15.113.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.15.113.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315935617919-15814-0/krb5kdc/oryx_127.15.113.129.keytab.
Entry for principal oryx/127.15.113.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315935617919-15814-0/krb5kdc/oryx_127.15.113.129.keytab.
I20260809 22:53:20.922137 15814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskItYPJR/build/asan/bin/kudu
/tmp/dist-test-taskItYPJR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315935617919-15814-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315935617919-15814-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315935617919-15814-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315935617919-15814-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.113.129:0
--local_ip_for_outbound_sockets=127.15.113.129
--webserver_interface=127.15.113.129
--webserver_port=0
--tserver_master_addrs=127.15.113.190:34881
--keytab_file=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315935617919-15814-0/krb5kdc/oryx_127.15.113.129.keytab
--principal=oryx/127.15.113.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.113.148:33227
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315935617919-15814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315935617919-15814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315935617919-15814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:53:21.276942 28236 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:53:21.277202 28236 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:53:21.277336 28236 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:53:21.284317 28236 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:53:21.284415 28236 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:53:21.284536 28236 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 22:53:21.284638 28236 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.113.129
I20260809 22:53:21.293967 28236 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.113.148:33227
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315935617919-15814-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315935617919-15814-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315935617919-15814-0/krb5kdc/oryx_127.15.113.129.keytab
--principal=oryx/127.15.113.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.113.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315935617919-15814-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.113.129
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.15.113.190:34881
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.113.129
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315935617919-15814-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:53:21.295090 28236 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:53:21.296350 28236 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:53:21.308600 28242 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:53:21.308602 28243 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:53:21.308652 28245 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:53:21.309680 28236 server_base.cc:1034] running on GCE node
I20260809 22:53:21.310236 28236 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:53:21.311039 28236 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:53:21.312270 28236 hybrid_clock.cc:648] HybridClock initialized: now 1786316001312210 us; error 40 us; skew 500 ppm
I20260809 22:53:21.312559 28236 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:53:21 dist-test-slave-ncp9 krb5kdc[28066](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316001, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.113.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:53:21.315692 28236 init.cc:377] Logged in from keytab as oryx/127.15.113.129@KRBTEST.COM (short username oryx)
I20260809 22:53:21.317060 28236 webserver.cc:460] Webserver started at http://127.15.113.129:43937/ using document root <none> and password file <none>
I20260809 22:53:21.317677 28236 fs_manager.cc:359] Metadata directory not provided
I20260809 22:53:21.317770 28236 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:53:21.318075 28236 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:53:21.320807 28236 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315935617919-15814-0/minicluster-data/ts-0/data/instance:
uuid: "b84b90efbbdb40bebe1b5adb43e9aa69"
format_stamp: "Formatted at 2026-08-09 22:53:21 on dist-test-slave-ncp9"
server_key: "25fbfbbd76e1e9a0f888ffa69ab63825"
server_key_iv: "71d808af6e5fcc973f52b8b5c846467d"
server_key_version: "encryptionkey@0"
I20260809 22:53:21.321605 28236 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315935617919-15814-0/minicluster-data/ts-0/wal/instance:
uuid: "b84b90efbbdb40bebe1b5adb43e9aa69"
format_stamp: "Formatted at 2026-08-09 22:53:21 on dist-test-slave-ncp9"
server_key: "25fbfbbd76e1e9a0f888ffa69ab63825"
server_key_iv: "71d808af6e5fcc973f52b8b5c846467d"
server_key_version: "encryptionkey@0"
I20260809 22:53:21.326231 28236 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.003s
I20260809 22:53:21.329432 28256 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:53:21.330683 28236 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260809 22:53:21.330801 28236 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315935617919-15814-0/minicluster-data/ts-0/data,/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315935617919-15814-0/minicluster-data/ts-0/wal
uuid: "b84b90efbbdb40bebe1b5adb43e9aa69"
format_stamp: "Formatted at 2026-08-09 22:53:21 on dist-test-slave-ncp9"
server_key: "25fbfbbd76e1e9a0f888ffa69ab63825"
server_key_iv: "71d808af6e5fcc973f52b8b5c846467d"
server_key_version: "encryptionkey@0"
I20260809 22:53:21.330968 28236 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315935617919-15814-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315935617919-15814-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315935617919-15814-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:53:21.356504 28236 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:53:21.360109 28236 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:53:21.360332 28236 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:53:21.362761 28236 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:53:21.362836 28236 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:53:21.362948 28236 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:53:21.363003 28236 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 22:53:21 dist-test-slave-ncp9 krb5kdc[28066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316001, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.113.129@KRBTEST.COM for oryx/127.15.113.190@KRBTEST.COM
I20260809 22:53:21.384577 28270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:21.363381 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.113.129:33553 (local address 127.15.113.190:34881)
0809 22:53:21.363872 (+ 491us) server_negotiation.cc:206] Beginning negotiation
0809 22:53:21.363878 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:53:21.364973 (+ 1095us) server_negotiation.cc:407] Connection header received
0809 22:53:21.367022 (+ 2049us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:53:21.367028 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:53:21.367119 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:53:21.367265 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:53:21.369428 (+ 2163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:21.370394 (+ 966us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:21.372843 (+ 2449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:21.373310 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:21.373430 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:21.378079 (+ 4649us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:53:21.378086 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:53:21.378090 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:53:21.378146 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:53:21.381234 (+ 3088us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:53:21.382165 (+ 931us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:53:21.382171 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:53:21.382174 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:53:21.382347 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:53:21.382883 (+ 536us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:53:21.382889 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:53:21.382893 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:53:21.383176 (+ 283us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:53:21.383316 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0809 22:53:21.384007 (+ 691us) server_negotiation.cc:299] Negotiation successful
0809 22:53:21.384232 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":358,"thread_start_us":246,"threads_started":1}
I20260809 22:53:21.385766 28271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:21.363942 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:34881 (local address 127.15.113.129:33553)
0809 22:53:21.364800 (+ 858us) negotiation.cc:107] Waiting for socket to connect
0809 22:53:21.364844 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 22:53:21.366750 (+ 1906us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:53:21.367549 (+ 799us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:53:21.367559 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:53:21.368298 (+ 739us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:53:21.369178 (+ 880us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:53:21.369209 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:53:21.370564 (+ 1355us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:53:21.370569 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:53:21.372697 (+ 2128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:53:21.372713 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:53:21.373576 (+ 863us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:53:21.373586 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:53:21.373837 (+ 251us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:21.374649 (+ 812us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:53:21.374673 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:53:21.377799 (+ 3126us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:53:21.381369 (+ 3570us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:53:21.381377 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:53:21.381401 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:53:21.382008 (+ 607us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:53:21.382505 (+ 497us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:53:21.382510 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:53:21.382513 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:53:21.382754 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:53:21.383308 (+ 554us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:53:21.383315 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:53:21.383634 (+ 319us) client_negotiation.cc:769] Sending connection context
0809 22:53:21.383861 (+ 227us) client_negotiation.cc:240] Negotiation successful
0809 22:53:21.384351 (+ 490us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":574,"thread_start_us":343,"threads_started":1}
I20260809 22:53:21.404605 28270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:21.391924 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.113.129:55061 (local address 127.15.113.190:34881)
0809 22:53:21.392069 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0809 22:53:21.392073 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 22:53:21.392609 (+ 536us) server_negotiation.cc:407] Connection header received
0809 22:53:21.392773 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:53:21.392778 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:53:21.392843 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:53:21.392948 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:53:21.393835 (+ 887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:21.394520 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:21.395821 (+ 1301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:21.396330 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:21.396453 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:21.398858 (+ 2405us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:53:21.398864 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:53:21.398869 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:53:21.398909 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:53:21.402007 (+ 3098us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:53:21.402667 (+ 660us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:53:21.402674 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:53:21.402678 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:53:21.402822 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:53:21.403288 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:53:21.403292 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:53:21.403294 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:53:21.403608 (+ 314us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:53:21.403765 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0809 22:53:21.404233 (+ 468us) server_negotiation.cc:299] Negotiation successful
0809 22:53:21.404389 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260809 22:53:21.404980 28381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:21.391944 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:34881 (local address 127.15.113.129:55061)
0809 22:53:21.392510 (+ 566us) negotiation.cc:107] Waiting for socket to connect
0809 22:53:21.392530 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 22:53:21.392658 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:53:21.392975 (+ 317us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:53:21.392979 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:53:21.393415 (+ 436us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:53:21.393700 (+ 285us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:53:21.393709 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:53:21.394651 (+ 942us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:53:21.394657 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:53:21.395684 (+ 1027us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:53:21.395699 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:53:21.396692 (+ 993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:53:21.396703 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:53:21.396925 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:21.397521 (+ 596us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:53:21.397544 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:53:21.398646 (+ 1102us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:53:21.402135 (+ 3489us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:53:21.402139 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:53:21.402141 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:53:21.402553 (+ 412us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:53:21.402950 (+ 397us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:53:21.402955 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:53:21.402959 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:53:21.403132 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:53:21.403749 (+ 617us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:53:21.403754 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:53:21.403965 (+ 211us) client_negotiation.cc:769] Sending connection context
0809 22:53:21.404135 (+ 170us) client_negotiation.cc:240] Negotiation successful
0809 22:53:21.404490 (+ 355us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":447,"thread_start_us":311,"threads_started":1}
I20260809 22:53:21.415917 28236 rpc_server.cc:307] RPC server started. Bound to: 127.15.113.129:42763
I20260809 22:53:21.415939 28437 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.113.129:42763 every 8 connection(s)
I20260809 22:53:21.417055 28236 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315935617919-15814-0/minicluster-data/ts-0/data/info.pb
I20260809 22:53:21.418915 28438 heartbeater.cc:347] Connected to a master server at 127.15.113.190:34881
I20260809 22:53:21.419107 28438 heartbeater.cc:464] Registering TS with master...
I20260809 22:53:21.419678 28438 heartbeater.cc:511] Master 127.15.113.190:34881 requested a full tablet report, sending...
I20260809 22:53:21.420910 15814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskItYPJR/build/asan/bin/kudu as pid 28236
I20260809 22:53:21.421092 15814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315935617919-15814-0/minicluster-data/ts-0/wal/instance
I20260809 22:53:21.421536 28128 ts_manager.cc:194] Registered new tserver with Master: b84b90efbbdb40bebe1b5adb43e9aa69 (127.15.113.129:42763)
I20260809 22:53:21.421730 15814 external_mini_cluster.cc:1428] Setting key 0fd1d1975ccbc38ad2a2d58cb09c120f
I20260809 22:53:21.423195 28128 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.15.113.129@KRBTEST.COM'} at 127.15.113.129:33553
WARNING: no policy specified for oryx/127.15.113.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.15.113.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.15.113.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315935617919-15814-0/krb5kdc/oryx_127.15.113.130.keytab.
Entry for principal oryx/127.15.113.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315935617919-15814-0/krb5kdc/oryx_127.15.113.130.keytab.
I20260809 22:53:21.477447 15814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskItYPJR/build/asan/bin/kudu
/tmp/dist-test-taskItYPJR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315935617919-15814-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315935617919-15814-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315935617919-15814-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315935617919-15814-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.113.130:0
--local_ip_for_outbound_sockets=127.15.113.130
--webserver_interface=127.15.113.130
--webserver_port=0
--tserver_master_addrs=127.15.113.190:34881
--keytab_file=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315935617919-15814-0/krb5kdc/oryx_127.15.113.130.keytab
--principal=oryx/127.15.113.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.113.148:33227
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315935617919-15814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315935617919-15814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315935617919-15814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:53:21.803761 28443 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:53:21.804010 28443 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:53:21.804142 28443 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:53:21.811321 28443 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:53:21.811422 28443 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:53:21.811566 28443 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 22:53:21.811668 28443 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.113.130
I20260809 22:53:21.821012 28443 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.113.148:33227
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315935617919-15814-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315935617919-15814-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315935617919-15814-0/krb5kdc/oryx_127.15.113.130.keytab
--principal=oryx/127.15.113.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.113.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315935617919-15814-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.113.130
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.15.113.190:34881
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.113.130
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315935617919-15814-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:53:21.822216 28443 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:53:21.823511 28443 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:53:21.835754 28449 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:53:21.835804 28452 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:53:21.835942 28454 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:53:21.836627 28443 server_base.cc:1034] running on GCE node
I20260809 22:53:21.837231 28443 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:53:21.838094 28443 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:53:21.839357 28443 hybrid_clock.cc:648] HybridClock initialized: now 1786316001839238 us; error 87 us; skew 500 ppm
I20260809 22:53:21.839701 28443 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:53:21 dist-test-slave-ncp9 krb5kdc[28066](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316001, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.113.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:53:21.842904 28443 init.cc:377] Logged in from keytab as oryx/127.15.113.130@KRBTEST.COM (short username oryx)
I20260809 22:53:21.844484 28443 webserver.cc:460] Webserver started at http://127.15.113.130:39677/ using document root <none> and password file <none>
I20260809 22:53:21.845260 28443 fs_manager.cc:359] Metadata directory not provided
I20260809 22:53:21.845376 28443 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:53:21.845743 28443 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:53:21.848973 28443 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315935617919-15814-0/minicluster-data/ts-1/data/instance:
uuid: "a85606f73667466f804f9495aab892b4"
format_stamp: "Formatted at 2026-08-09 22:53:21 on dist-test-slave-ncp9"
server_key: "a3a2011dc712562761fe000121a61239"
server_key_iv: "1ebe729925311b0d6d2c74c48f8e7f02"
server_key_version: "encryptionkey@0"
I20260809 22:53:21.849846 28443 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315935617919-15814-0/minicluster-data/ts-1/wal/instance:
uuid: "a85606f73667466f804f9495aab892b4"
format_stamp: "Formatted at 2026-08-09 22:53:21 on dist-test-slave-ncp9"
server_key: "a3a2011dc712562761fe000121a61239"
server_key_iv: "1ebe729925311b0d6d2c74c48f8e7f02"
server_key_version: "encryptionkey@0"
I20260809 22:53:21.855197 28443 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.003s
I20260809 22:53:21.858908 28467 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:53:21.860188 28443 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 22:53:21.860345 28443 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315935617919-15814-0/minicluster-data/ts-1/data,/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315935617919-15814-0/minicluster-data/ts-1/wal
uuid: "a85606f73667466f804f9495aab892b4"
format_stamp: "Formatted at 2026-08-09 22:53:21 on dist-test-slave-ncp9"
server_key: "a3a2011dc712562761fe000121a61239"
server_key_iv: "1ebe729925311b0d6d2c74c48f8e7f02"
server_key_version: "encryptionkey@0"
I20260809 22:53:21.860570 28443 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315935617919-15814-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315935617919-15814-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315935617919-15814-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:53:21.891029 28443 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:53:21.895623 28443 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:53:21.895891 28443 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:53:21.898620 28443 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:53:21.898705 28443 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:53:21.898849 28443 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:53:21.898924 28443 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 22:53:21 dist-test-slave-ncp9 krb5kdc[28066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316001, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.113.130@KRBTEST.COM for oryx/127.15.113.190@KRBTEST.COM
I20260809 22:53:21.919831 28270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:21.899049 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.113.130:56213 (local address 127.15.113.190:34881)
0809 22:53:21.899268 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0809 22:53:21.899276 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 22:53:21.900650 (+ 1374us) server_negotiation.cc:407] Connection header received
0809 22:53:21.902723 (+ 2073us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:53:21.902729 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:53:21.902807 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:53:21.902896 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:53:21.905015 (+ 2119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:21.906017 (+ 1002us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:21.908741 (+ 2724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:21.909286 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:21.909404 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:21.913011 (+ 3607us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:53:21.913018 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:53:21.913023 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:53:21.913059 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:53:21.916358 (+ 3299us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:53:21.917232 (+ 874us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:53:21.917237 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:53:21.917240 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:53:21.917394 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:53:21.917995 (+ 601us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:53:21.918000 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:53:21.918003 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:53:21.918323 (+ 320us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:53:21.918484 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0809 22:53:21.919272 (+ 788us) server_negotiation.cc:299] Negotiation successful
0809 22:53:21.919491 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260809 22:53:21.920739 28484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:21.899673 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:34881 (local address 127.15.113.130:56213)
0809 22:53:21.900500 (+ 827us) negotiation.cc:107] Waiting for socket to connect
0809 22:53:21.900546 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0809 22:53:21.902460 (+ 1914us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:53:21.903200 (+ 740us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:53:21.903213 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:53:21.903932 (+ 719us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:53:21.904749 (+ 817us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:53:21.904779 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:53:21.906209 (+ 1430us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:53:21.906219 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:53:21.908601 (+ 2382us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:53:21.908617 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:53:21.909539 (+ 922us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:53:21.909548 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:53:21.909742 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:21.910352 (+ 610us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:53:21.910372 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:53:21.912704 (+ 2332us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:53:21.916473 (+ 3769us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:53:21.916485 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:53:21.916499 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:53:21.917120 (+ 621us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:53:21.917538 (+ 418us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:53:21.917543 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:53:21.917547 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:53:21.917820 (+ 273us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:53:21.918563 (+ 743us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:53:21.918570 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:53:21.918908 (+ 338us) client_negotiation.cc:769] Sending connection context
0809 22:53:21.919127 (+ 219us) client_negotiation.cc:240] Negotiation successful
0809 22:53:21.919593 (+ 466us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":566,"thread_start_us":343,"threads_started":1}
I20260809 22:53:21.938944 28270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:21.926589 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.113.130:59605 (local address 127.15.113.190:34881)
0809 22:53:21.926770 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0809 22:53:21.926776 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:53:21.926851 (+ 75us) server_negotiation.cc:407] Connection header received
0809 22:53:21.926983 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:53:21.926988 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:53:21.927064 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:53:21.927172 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:53:21.927981 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:21.929192 (+ 1211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:21.930442 (+ 1250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:21.931113 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:21.931236 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:21.932838 (+ 1602us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:53:21.932844 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:53:21.932849 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:53:21.932885 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:53:21.936390 (+ 3505us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:53:21.937050 (+ 660us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:53:21.937056 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:53:21.937059 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:53:21.937219 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:53:21.937556 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:53:21.937562 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:53:21.937565 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:53:21.937933 (+ 368us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:53:21.938080 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0809 22:53:21.938543 (+ 463us) server_negotiation.cc:299] Negotiation successful
0809 22:53:21.938703 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260809 22:53:21.939258 28585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:21.926128 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:34881 (local address 127.15.113.130:59605)
0809 22:53:21.926771 (+ 643us) negotiation.cc:107] Waiting for socket to connect
0809 22:53:21.926787 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 22:53:21.926886 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:53:21.927180 (+ 294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:53:21.927182 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:53:21.927568 (+ 386us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:53:21.927862 (+ 294us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:53:21.927870 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:53:21.929316 (+ 1446us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:53:21.929320 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:53:21.930326 (+ 1006us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:53:21.930338 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:53:21.931265 (+ 927us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:53:21.931269 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:53:21.931395 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:21.931943 (+ 548us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:53:21.931958 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:53:21.932631 (+ 673us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:53:21.936506 (+ 3875us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:53:21.936510 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:53:21.936513 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:53:21.936873 (+ 360us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:53:21.937317 (+ 444us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:53:21.937325 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:53:21.937328 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:53:21.937445 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:53:21.938089 (+ 644us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:53:21.938098 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:53:21.938309 (+ 211us) client_negotiation.cc:769] Sending connection context
0809 22:53:21.938471 (+ 162us) client_negotiation.cc:240] Negotiation successful
0809 22:53:21.938799 (+ 328us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":492,"thread_start_us":339,"threads_started":1}
I20260809 22:53:21.953146 28443 rpc_server.cc:307] RPC server started. Bound to: 127.15.113.130:46333
I20260809 22:53:21.953168 28660 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.113.130:46333 every 8 connection(s)
I20260809 22:53:21.954284 28443 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315935617919-15814-0/minicluster-data/ts-1/data/info.pb
I20260809 22:53:21.956043 28661 heartbeater.cc:347] Connected to a master server at 127.15.113.190:34881
I20260809 22:53:21.956252 28661 heartbeater.cc:464] Registering TS with master...
I20260809 22:53:21.956854 28661 heartbeater.cc:511] Master 127.15.113.190:34881 requested a full tablet report, sending...
I20260809 22:53:21.958135 28128 ts_manager.cc:194] Registered new tserver with Master: a85606f73667466f804f9495aab892b4 (127.15.113.130:46333)
I20260809 22:53:21.959236 28128 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.15.113.130@KRBTEST.COM'} at 127.15.113.130:56213
I20260809 22:53:21.959462 15814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskItYPJR/build/asan/bin/kudu as pid 28443
I20260809 22:53:21.959671 15814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315935617919-15814-0/minicluster-data/ts-1/wal/instance
I20260809 22:53:21.960101 15814 external_mini_cluster.cc:1428] Setting key 89882b37ed387c0d4bd42a2b0b8c3813
WARNING: no policy specified for oryx/127.15.113.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.15.113.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.15.113.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315935617919-15814-0/krb5kdc/oryx_127.15.113.131.keytab.
Entry for principal oryx/127.15.113.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315935617919-15814-0/krb5kdc/oryx_127.15.113.131.keytab.
I20260809 22:53:22.012025 15814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskItYPJR/build/asan/bin/kudu
/tmp/dist-test-taskItYPJR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315935617919-15814-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315935617919-15814-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315935617919-15814-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315935617919-15814-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.113.131:0
--local_ip_for_outbound_sockets=127.15.113.131
--webserver_interface=127.15.113.131
--webserver_port=0
--tserver_master_addrs=127.15.113.190:34881
--keytab_file=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315935617919-15814-0/krb5kdc/oryx_127.15.113.131.keytab
--principal=oryx/127.15.113.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.113.148:33227
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315935617919-15814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315935617919-15814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315935617919-15814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:53:22.326997 28666 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:53:22.327243 28666 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:53:22.327345 28666 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:53:22.334321 28666 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:53:22.334411 28666 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:53:22.334494 28666 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 22:53:22.334570 28666 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.113.131
I20260809 22:53:22.343647 28666 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.113.148:33227
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315935617919-15814-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315935617919-15814-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315935617919-15814-0/krb5kdc/oryx_127.15.113.131.keytab
--principal=oryx/127.15.113.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.113.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315935617919-15814-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.113.131
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.15.113.190:34881
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.113.131
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315935617919-15814-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:53:22.344674 28666 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:53:22.345937 28666 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:53:22.357872 28682 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:53:22.357918 28675 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:53:22.357944 28677 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:53:22.358848 28666 server_base.cc:1034] running on GCE node
I20260809 22:53:22.359455 28666 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:53:22.360322 28666 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:53:22.361534 28666 hybrid_clock.cc:648] HybridClock initialized: now 1786316002361469 us; error 35 us; skew 500 ppm
I20260809 22:53:22.361807 28666 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:53:22 dist-test-slave-ncp9 krb5kdc[28066](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316002, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.113.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:53:22.364754 28666 init.cc:377] Logged in from keytab as oryx/127.15.113.131@KRBTEST.COM (short username oryx)
I20260809 22:53:22.366169 28666 webserver.cc:460] Webserver started at http://127.15.113.131:40385/ using document root <none> and password file <none>
I20260809 22:53:22.366798 28666 fs_manager.cc:359] Metadata directory not provided
I20260809 22:53:22.366891 28666 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:53:22.367200 28666 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:53:22.369990 28666 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315935617919-15814-0/minicluster-data/ts-2/data/instance:
uuid: "ef758642c9de4d939a89e3d634d6a636"
format_stamp: "Formatted at 2026-08-09 22:53:22 on dist-test-slave-ncp9"
server_key: "dc4be29e138be960b9c85ab252e1fb86"
server_key_iv: "8e2da6aed8b87b66f173d831a51fb28c"
server_key_version: "encryptionkey@0"
I20260809 22:53:22.370784 28666 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315935617919-15814-0/minicluster-data/ts-2/wal/instance:
uuid: "ef758642c9de4d939a89e3d634d6a636"
format_stamp: "Formatted at 2026-08-09 22:53:22 on dist-test-slave-ncp9"
server_key: "dc4be29e138be960b9c85ab252e1fb86"
server_key_iv: "8e2da6aed8b87b66f173d831a51fb28c"
server_key_version: "encryptionkey@0"
I20260809 22:53:22.375526 28666 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260809 22:53:22.378732 28689 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:53:22.379942 28666 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 22:53:22.380090 28666 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315935617919-15814-0/minicluster-data/ts-2/data,/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315935617919-15814-0/minicluster-data/ts-2/wal
uuid: "ef758642c9de4d939a89e3d634d6a636"
format_stamp: "Formatted at 2026-08-09 22:53:22 on dist-test-slave-ncp9"
server_key: "dc4be29e138be960b9c85ab252e1fb86"
server_key_iv: "8e2da6aed8b87b66f173d831a51fb28c"
server_key_version: "encryptionkey@0"
I20260809 22:53:22.380295 28666 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315935617919-15814-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315935617919-15814-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315935617919-15814-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:53:22.418376 28666 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:53:22.422420 28666 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:53:22.422668 28666 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:53:22.425223 28666 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:53:22.425324 28666 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:53:22.425452 28666 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:53:22.425520 28666 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:53:22.425838 28438 heartbeater.cc:503] Master 127.15.113.190:34881 was elected leader, sending a full tablet report...
Aug 09 22:53:22 dist-test-slave-ncp9 krb5kdc[28066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316002, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.113.131@KRBTEST.COM for oryx/127.15.113.190@KRBTEST.COM
I20260809 22:53:22.447867 28270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:22.425792 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.113.131:47519 (local address 127.15.113.190:34881)
0809 22:53:22.425979 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0809 22:53:22.425985 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:53:22.427552 (+ 1567us) server_negotiation.cc:407] Connection header received
0809 22:53:22.429810 (+ 2258us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:53:22.429816 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:53:22.429899 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:53:22.429979 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:53:22.431799 (+ 1820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:22.432790 (+ 991us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:22.435739 (+ 2949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:22.436374 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:22.436520 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:22.441023 (+ 4503us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:53:22.441030 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:53:22.441035 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:53:22.441078 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:53:22.444561 (+ 3483us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:53:22.445439 (+ 878us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:53:22.445445 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:53:22.445448 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:53:22.445593 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:53:22.446133 (+ 540us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:53:22.446138 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:53:22.446142 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:53:22.446454 (+ 312us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:53:22.446608 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0809 22:53:22.447458 (+ 850us) server_negotiation.cc:299] Negotiation successful
0809 22:53:22.447623 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260809 22:53:22.449434 28704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:22.426479 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:34881 (local address 127.15.113.131:47519)
0809 22:53:22.427403 (+ 924us) negotiation.cc:107] Waiting for socket to connect
0809 22:53:22.427458 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0809 22:53:22.429522 (+ 2064us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:53:22.430244 (+ 722us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:53:22.430251 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:53:22.430843 (+ 592us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:53:22.431622 (+ 779us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:53:22.431656 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:53:22.433002 (+ 1346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:53:22.433009 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:53:22.435535 (+ 2526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:53:22.435550 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:53:22.436613 (+ 1063us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:53:22.436621 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:53:22.436857 (+ 236us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:22.437568 (+ 711us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:53:22.437596 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:53:22.440770 (+ 3174us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:53:22.444723 (+ 3953us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:53:22.444729 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:53:22.444739 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:53:22.445319 (+ 580us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:53:22.445756 (+ 437us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:53:22.445760 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:53:22.445764 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:53:22.446008 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:53:22.446600 (+ 592us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:53:22.446611 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:53:22.447062 (+ 451us) client_negotiation.cc:769] Sending connection context
0809 22:53:22.447364 (+ 302us) client_negotiation.cc:240] Negotiation successful
0809 22:53:22.447890 (+ 526us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":653,"thread_start_us":375,"threads_started":1}
I20260809 22:53:22.467690 28270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:22.454508 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.113.131:43173 (local address 127.15.113.190:34881)
0809 22:53:22.454646 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0809 22:53:22.454651 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:53:22.455234 (+ 583us) server_negotiation.cc:407] Connection header received
0809 22:53:22.455335 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:53:22.455341 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:53:22.455434 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:53:22.455537 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:53:22.456346 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:22.457466 (+ 1120us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:22.459231 (+ 1765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:22.459639 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:22.459760 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:22.461520 (+ 1760us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:53:22.461526 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:53:22.461531 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:53:22.461566 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:53:22.464827 (+ 3261us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:53:22.465650 (+ 823us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:53:22.465655 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:53:22.465658 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:53:22.465808 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:53:22.466133 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:53:22.466137 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:53:22.466140 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:53:22.466503 (+ 363us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:53:22.466663 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0809 22:53:22.467137 (+ 474us) server_negotiation.cc:299] Negotiation successful
0809 22:53:22.467342 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260809 22:53:22.467774 28798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:22.454575 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:34881 (local address 127.15.113.131:43173)
0809 22:53:22.455127 (+ 552us) negotiation.cc:107] Waiting for socket to connect
0809 22:53:22.455142 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 22:53:22.455250 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:53:22.455542 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:53:22.455545 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:53:22.455951 (+ 406us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:53:22.456220 (+ 269us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:53:22.456229 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:53:22.457612 (+ 1383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:53:22.457618 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:53:22.459079 (+ 1461us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:53:22.459096 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:53:22.459814 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:53:22.459821 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:53:22.459965 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:22.460497 (+ 532us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:53:22.460512 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:53:22.461278 (+ 766us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:53:22.465114 (+ 3836us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:53:22.465119 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:53:22.465123 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:53:22.465529 (+ 406us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:53:22.465916 (+ 387us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:53:22.465920 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:53:22.465922 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:53:22.466038 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:53:22.466643 (+ 605us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:53:22.466659 (+ 16us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:53:22.466863 (+ 204us) client_negotiation.cc:769] Sending connection context
0809 22:53:22.467038 (+ 175us) client_negotiation.cc:240] Negotiation successful
0809 22:53:22.467346 (+ 308us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":442,"thread_start_us":310,"threads_started":1}
I20260809 22:53:22.477100 28666 rpc_server.cc:307] RPC server started. Bound to: 127.15.113.131:41323
I20260809 22:53:22.477128 28864 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.113.131:41323 every 8 connection(s)
I20260809 22:53:22.478238 28666 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315935617919-15814-0/minicluster-data/ts-2/data/info.pb
I20260809 22:53:22.480103 28865 heartbeater.cc:347] Connected to a master server at 127.15.113.190:34881
I20260809 22:53:22.480342 28865 heartbeater.cc:464] Registering TS with master...
I20260809 22:53:22.481076 28865 heartbeater.cc:511] Master 127.15.113.190:34881 requested a full tablet report, sending...
I20260809 22:53:22.482034 15814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskItYPJR/build/asan/bin/kudu as pid 28666
I20260809 22:53:22.482225 15814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315935617919-15814-0/minicluster-data/ts-2/wal/instance
I20260809 22:53:22.482481 28128 ts_manager.cc:194] Registered new tserver with Master: ef758642c9de4d939a89e3d634d6a636 (127.15.113.131:41323)
I20260809 22:53:22.482757 15814 external_mini_cluster.cc:1428] Setting key f661c8b439a1c34a93e2709878cbd1ac
I20260809 22:53:22.483513 28128 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.15.113.131@KRBTEST.COM'} at 127.15.113.131:47519
I20260809 22:53:22.485632 15814 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 09 22:53:22 dist-test-slave-ncp9 krb5kdc[28066](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316002, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 09 22:53:22 dist-test-slave-ncp9 krb5kdc[28066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.15.113.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 22:53:22 dist-test-slave-ncp9 krb5kdc[28066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.15.113.190@KRBTEST.COM, Server not found in Kerberos database
W20260809 22:53:22.542671 28270 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:53:22.532207 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49118 (local address 127.15.113.190:34881)
0809 22:53:22.532428 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0809 22:53:22.532434 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:53:22.532625 (+ 191us) server_negotiation.cc:407] Connection header received
0809 22:53:22.532827 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:53:22.532832 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:53:22.532920 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:53:22.533011 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:53:22.534372 (+ 1361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:22.535225 (+ 853us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:22.536465 (+ 1240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:22.536924 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:22.537041 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:22.542448 (+ 5407us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49118: BlockingRecv error: recv got EOF from 127.0.0.1:49118 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
Aug 09 22:53:22 dist-test-slave-ncp9 krb5kdc[28066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316002, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.15.113.190@KRBTEST.COM
I20260809 22:53:22.558086 28270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:22.545426 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49134 (local address 127.15.113.190:34881)
0809 22:53:22.545613 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0809 22:53:22.545619 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:53:22.545889 (+ 270us) server_negotiation.cc:407] Connection header received
0809 22:53:22.546047 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:53:22.546052 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:53:22.546139 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:53:22.546221 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:53:22.547307 (+ 1086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:22.548031 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:22.549276 (+ 1245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:22.549701 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:22.549810 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:22.553409 (+ 3599us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:53:22.553414 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:53:22.553417 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:53:22.553451 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:53:22.555648 (+ 2197us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:53:22.556246 (+ 598us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:53:22.556251 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:53:22.556254 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:53:22.556375 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:53:22.556724 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:53:22.556729 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:53:22.556732 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:53:22.557048 (+ 316us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:53:22.557351 (+ 303us) server_negotiation.cc:1035] Waiting for connection context
0809 22:53:22.557578 (+ 227us) server_negotiation.cc:299] Negotiation successful
0809 22:53:22.557787 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260809 22:53:22.561728 28127 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:49134:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 22:53:22.568454 28127 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 22:53:22.599272 28898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:22.580986 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37298 (local address 127.15.113.130:46333)
0809 22:53:22.589658 (+ 8672us) server_negotiation.cc:206] Beginning negotiation
0809 22:53:22.589669 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 22:53:22.591331 (+ 1662us) server_negotiation.cc:407] Connection header received
0809 22:53:22.591528 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:53:22.591536 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:53:22.591824 (+ 288us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:53:22.592019 (+ 195us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:53:22.592988 (+ 969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:22.594418 (+ 1430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:22.596825 (+ 2407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:22.598116 (+ 1291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:22.598323 (+ 207us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:22.598495 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0809 22:53:22.598799 (+ 304us) server_negotiation.cc:299] Negotiation successful
0809 22:53:22.598972 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8479,"thread_start_us":8345,"threads_started":1}
I20260809 22:53:22.599344 28896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:22.580258 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.130:46333 (local address 127.0.0.1:37298)
0809 22:53:22.591222 (+ 10964us) negotiation.cc:107] Waiting for socket to connect
0809 22:53:22.591240 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 22:53:22.591380 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:53:22.592271 (+ 891us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:53:22.592278 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:53:22.592307 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:53:22.592720 (+ 413us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:53:22.592735 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:53:22.594555 (+ 1820us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:53:22.594561 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:53:22.596508 (+ 1947us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:53:22.596523 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:53:22.598358 (+ 1835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:53:22.598365 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:53:22.598547 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:22.598597 (+ 50us) client_negotiation.cc:769] Sending connection context
0809 22:53:22.598746 (+ 149us) client_negotiation.cc:240] Negotiation successful
0809 22:53:22.598932 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10814,"mutex_wait_us":47,"thread_start_us":9831,"threads_started":1}
I20260809 22:53:22.599917 28893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:22.580075 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.129:42763 (local address 127.0.0.1:35596)
0809 22:53:22.590435 (+ 10360us) negotiation.cc:107] Waiting for socket to connect
0809 22:53:22.590480 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0809 22:53:22.590638 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:53:22.592875 (+ 2237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:53:22.592879 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:53:22.592934 (+ 55us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:53:22.593613 (+ 679us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:53:22.593637 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:53:22.595152 (+ 1515us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:53:22.595158 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:53:22.597342 (+ 2184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:53:22.597358 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:53:22.599209 (+ 1851us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:53:22.599215 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:53:22.599383 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:22.599413 (+ 30us) client_negotiation.cc:769] Sending connection context
0809 22:53:22.599507 (+ 94us) client_negotiation.cc:240] Negotiation successful
0809 22:53:22.599624 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10136,"thread_start_us":11388,"threads_started":1}
I20260809 22:53:22.600374 28899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:22.581429 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49762 (local address 127.15.113.131:41323)
0809 22:53:22.592131 (+ 10702us) server_negotiation.cc:206] Beginning negotiation
0809 22:53:22.592137 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:53:22.592164 (+ 27us) server_negotiation.cc:407] Connection header received
0809 22:53:22.592236 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:53:22.592241 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:53:22.592397 (+ 156us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:53:22.592554 (+ 157us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:53:22.593799 (+ 1245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:22.595214 (+ 1415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:22.597704 (+ 2490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:22.598997 (+ 1293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:22.599181 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:22.599336 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0809 22:53:22.599885 (+ 549us) server_negotiation.cc:299] Negotiation successful
0809 22:53:22.600050 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10548,"thread_start_us":10390,"threads_started":1}
I20260809 22:53:22.600457 28891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:22.580558 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.131:41323 (local address 127.0.0.1:49762)
0809 22:53:22.591518 (+ 10960us) negotiation.cc:107] Waiting for socket to connect
0809 22:53:22.591539 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0809 22:53:22.591662 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:53:22.592567 (+ 905us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:53:22.592572 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:53:22.592595 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:53:22.593322 (+ 727us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:53:22.593342 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:53:22.595366 (+ 2024us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:53:22.595372 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:53:22.597536 (+ 2164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:53:22.597549 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:53:22.599499 (+ 1950us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:53:22.599504 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:53:22.599676 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:22.599706 (+ 30us) client_negotiation.cc:769] Sending connection context
0809 22:53:22.599797 (+ 91us) client_negotiation.cc:240] Negotiation successful
0809 22:53:22.599914 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10853,"mutex_wait_us":1,"thread_start_us":10358,"threads_started":1}
I20260809 22:53:22.600472 28892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:22.580238 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35596 (local address 127.15.113.129:42763)
0809 22:53:22.592121 (+ 11883us) server_negotiation.cc:206] Beginning negotiation
0809 22:53:22.592129 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 22:53:22.592157 (+ 28us) server_negotiation.cc:407] Connection header received
0809 22:53:22.592257 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:53:22.592265 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:53:22.592505 (+ 240us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:53:22.592711 (+ 206us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:53:22.593831 (+ 1120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:22.595006 (+ 1175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:22.597535 (+ 2529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:22.598744 (+ 1209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:22.598894 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:22.599048 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0809 22:53:22.600043 (+ 995us) server_negotiation.cc:299] Negotiation successful
0809 22:53:22.600197 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11729,"thread_start_us":12644,"threads_started":1}
I20260809 22:53:22.603530 28765 tablet_service.cc:1467] Processing CreateTablet for tablet 5450590aa15f4d8998b0eba02a00f411 (DEFAULT_TABLE table=test-table [id=4310a815351a4642a007c824068f2334]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:53:22.603555 28549 tablet_service.cc:1467] Processing CreateTablet for tablet 5450590aa15f4d8998b0eba02a00f411 (DEFAULT_TABLE table=test-table [id=4310a815351a4642a007c824068f2334]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:53:22.604745 28765 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5450590aa15f4d8998b0eba02a00f411. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:53:22.604916 28549 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5450590aa15f4d8998b0eba02a00f411. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:53:22.604581 28330 tablet_service.cc:1467] Processing CreateTablet for tablet 5450590aa15f4d8998b0eba02a00f411 (DEFAULT_TABLE table=test-table [id=4310a815351a4642a007c824068f2334]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:53:22.605856 28330 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5450590aa15f4d8998b0eba02a00f411. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:53:22.615819 28900 tablet_bootstrap.cc:492] T 5450590aa15f4d8998b0eba02a00f411 P ef758642c9de4d939a89e3d634d6a636: Bootstrap starting.
I20260809 22:53:22.619009 28901 tablet_bootstrap.cc:492] T 5450590aa15f4d8998b0eba02a00f411 P a85606f73667466f804f9495aab892b4: Bootstrap starting.
I20260809 22:53:22.620118 28902 tablet_bootstrap.cc:492] T 5450590aa15f4d8998b0eba02a00f411 P b84b90efbbdb40bebe1b5adb43e9aa69: Bootstrap starting.
I20260809 22:53:22.620365 28900 tablet_bootstrap.cc:654] T 5450590aa15f4d8998b0eba02a00f411 P ef758642c9de4d939a89e3d634d6a636: Neither blocks nor log segments found. Creating new log.
I20260809 22:53:22.621726 28900 log.cc:824] T 5450590aa15f4d8998b0eba02a00f411 P ef758642c9de4d939a89e3d634d6a636: Log is configured to *not* fsync() on all Append() calls
I20260809 22:53:22.622454 28901 tablet_bootstrap.cc:654] T 5450590aa15f4d8998b0eba02a00f411 P a85606f73667466f804f9495aab892b4: Neither blocks nor log segments found. Creating new log.
I20260809 22:53:22.623211 28902 tablet_bootstrap.cc:654] T 5450590aa15f4d8998b0eba02a00f411 P b84b90efbbdb40bebe1b5adb43e9aa69: Neither blocks nor log segments found. Creating new log.
I20260809 22:53:22.623562 28901 log.cc:824] T 5450590aa15f4d8998b0eba02a00f411 P a85606f73667466f804f9495aab892b4: Log is configured to *not* fsync() on all Append() calls
I20260809 22:53:22.624328 28902 log.cc:824] T 5450590aa15f4d8998b0eba02a00f411 P b84b90efbbdb40bebe1b5adb43e9aa69: Log is configured to *not* fsync() on all Append() calls
I20260809 22:53:22.624797 28900 tablet_bootstrap.cc:492] T 5450590aa15f4d8998b0eba02a00f411 P ef758642c9de4d939a89e3d634d6a636: No bootstrap required, opened a new log
I20260809 22:53:22.625108 28900 ts_tablet_manager.cc:1397] T 5450590aa15f4d8998b0eba02a00f411 P ef758642c9de4d939a89e3d634d6a636: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260809 22:53:22.626468 28901 tablet_bootstrap.cc:492] T 5450590aa15f4d8998b0eba02a00f411 P a85606f73667466f804f9495aab892b4: No bootstrap required, opened a new log
I20260809 22:53:22.626731 28901 ts_tablet_manager.cc:1397] T 5450590aa15f4d8998b0eba02a00f411 P a85606f73667466f804f9495aab892b4: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.000s
I20260809 22:53:22.633282 28902 tablet_bootstrap.cc:492] T 5450590aa15f4d8998b0eba02a00f411 P b84b90efbbdb40bebe1b5adb43e9aa69: No bootstrap required, opened a new log
I20260809 22:53:22.633504 28902 ts_tablet_manager.cc:1397] T 5450590aa15f4d8998b0eba02a00f411 P b84b90efbbdb40bebe1b5adb43e9aa69: Time spent bootstrapping tablet: real 0.014s user 0.012s sys 0.000s
I20260809 22:53:22.636029 28900 raft_consensus.cc:348] T 5450590aa15f4d8998b0eba02a00f411 P ef758642c9de4d939a89e3d634d6a636 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b84b90efbbdb40bebe1b5adb43e9aa69" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 42763 } } peers { permanent_uuid: "a85606f73667466f804f9495aab892b4" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 46333 } } peers { permanent_uuid: "ef758642c9de4d939a89e3d634d6a636" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 41323 } }
I20260809 22:53:22.636451 28900 raft_consensus.cc:374] T 5450590aa15f4d8998b0eba02a00f411 P ef758642c9de4d939a89e3d634d6a636 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:53:22.636556 28900 raft_consensus.cc:729] T 5450590aa15f4d8998b0eba02a00f411 P ef758642c9de4d939a89e3d634d6a636 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef758642c9de4d939a89e3d634d6a636, State: Initialized, Role: FOLLOWER
I20260809 22:53:22.636721 28901 raft_consensus.cc:348] T 5450590aa15f4d8998b0eba02a00f411 P a85606f73667466f804f9495aab892b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b84b90efbbdb40bebe1b5adb43e9aa69" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 42763 } } peers { permanent_uuid: "a85606f73667466f804f9495aab892b4" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 46333 } } peers { permanent_uuid: "ef758642c9de4d939a89e3d634d6a636" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 41323 } }
I20260809 22:53:22.637166 28901 raft_consensus.cc:374] T 5450590aa15f4d8998b0eba02a00f411 P a85606f73667466f804f9495aab892b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:53:22.637274 28901 raft_consensus.cc:729] T 5450590aa15f4d8998b0eba02a00f411 P a85606f73667466f804f9495aab892b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a85606f73667466f804f9495aab892b4, State: Initialized, Role: FOLLOWER
I20260809 22:53:22.637219 28900 consensus_queue.cc:260] T 5450590aa15f4d8998b0eba02a00f411 P ef758642c9de4d939a89e3d634d6a636 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b84b90efbbdb40bebe1b5adb43e9aa69" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 42763 } } peers { permanent_uuid: "a85606f73667466f804f9495aab892b4" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 46333 } } peers { permanent_uuid: "ef758642c9de4d939a89e3d634d6a636" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 41323 } }
I20260809 22:53:22.637871 28901 consensus_queue.cc:260] T 5450590aa15f4d8998b0eba02a00f411 P a85606f73667466f804f9495aab892b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b84b90efbbdb40bebe1b5adb43e9aa69" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 42763 } } peers { permanent_uuid: "a85606f73667466f804f9495aab892b4" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 46333 } } peers { permanent_uuid: "ef758642c9de4d939a89e3d634d6a636" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 41323 } }
I20260809 22:53:22.638995 28865 heartbeater.cc:503] Master 127.15.113.190:34881 was elected leader, sending a full tablet report...
I20260809 22:53:22.641434 28900 ts_tablet_manager.cc:1428] T 5450590aa15f4d8998b0eba02a00f411 P ef758642c9de4d939a89e3d634d6a636: Time spent starting tablet: real 0.016s user 0.013s sys 0.000s
I20260809 22:53:22.641377 28902 raft_consensus.cc:348] T 5450590aa15f4d8998b0eba02a00f411 P b84b90efbbdb40bebe1b5adb43e9aa69 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b84b90efbbdb40bebe1b5adb43e9aa69" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 42763 } } peers { permanent_uuid: "a85606f73667466f804f9495aab892b4" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 46333 } } peers { permanent_uuid: "ef758642c9de4d939a89e3d634d6a636" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 41323 } }
I20260809 22:53:22.641742 28902 raft_consensus.cc:374] T 5450590aa15f4d8998b0eba02a00f411 P b84b90efbbdb40bebe1b5adb43e9aa69 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:53:22.641830 28902 raft_consensus.cc:729] T 5450590aa15f4d8998b0eba02a00f411 P b84b90efbbdb40bebe1b5adb43e9aa69 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b84b90efbbdb40bebe1b5adb43e9aa69, State: Initialized, Role: FOLLOWER
I20260809 22:53:22.642458 28902 consensus_queue.cc:260] T 5450590aa15f4d8998b0eba02a00f411 P b84b90efbbdb40bebe1b5adb43e9aa69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b84b90efbbdb40bebe1b5adb43e9aa69" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 42763 } } peers { permanent_uuid: "a85606f73667466f804f9495aab892b4" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 46333 } } peers { permanent_uuid: "ef758642c9de4d939a89e3d634d6a636" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 41323 } }
I20260809 22:53:22.644848 28661 heartbeater.cc:503] Master 127.15.113.190:34881 was elected leader, sending a full tablet report...
I20260809 22:53:22.645802 28901 ts_tablet_manager.cc:1428] T 5450590aa15f4d8998b0eba02a00f411 P a85606f73667466f804f9495aab892b4: Time spent starting tablet: real 0.019s user 0.011s sys 0.008s
I20260809 22:53:22.647585 28902 ts_tablet_manager.cc:1428] T 5450590aa15f4d8998b0eba02a00f411 P b84b90efbbdb40bebe1b5adb43e9aa69: Time spent starting tablet: real 0.014s user 0.006s sys 0.007s
W20260809 22:53:22.669102 28439 tablet.cc:2369] T 5450590aa15f4d8998b0eba02a00f411 P b84b90efbbdb40bebe1b5adb43e9aa69: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 22:53:22.706111 28663 tablet.cc:2369] T 5450590aa15f4d8998b0eba02a00f411 P a85606f73667466f804f9495aab892b4: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 22:53:22.729694 28867 tablet.cc:2369] T 5450590aa15f4d8998b0eba02a00f411 P ef758642c9de4d939a89e3d634d6a636: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 22:53:22.903460 28908 raft_consensus.cc:482] T 5450590aa15f4d8998b0eba02a00f411 P ef758642c9de4d939a89e3d634d6a636 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 22:53:22.903782 28908 raft_consensus.cc:504] T 5450590aa15f4d8998b0eba02a00f411 P ef758642c9de4d939a89e3d634d6a636 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b84b90efbbdb40bebe1b5adb43e9aa69" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 42763 } } peers { permanent_uuid: "a85606f73667466f804f9495aab892b4" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 46333 } } peers { permanent_uuid: "ef758642c9de4d939a89e3d634d6a636" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 41323 } }
I20260809 22:53:22.905567 28908 leader_election.cc:288] T 5450590aa15f4d8998b0eba02a00f411 P ef758642c9de4d939a89e3d634d6a636 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b84b90efbbdb40bebe1b5adb43e9aa69 (127.15.113.129:42763), a85606f73667466f804f9495aab892b4 (127.15.113.130:46333)
I20260809 22:53:22.911907 28704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:22.905858 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.129:42763 (local address 127.15.113.131:45401)
0809 22:53:22.906235 (+ 377us) negotiation.cc:107] Waiting for socket to connect
0809 22:53:22.906264 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0809 22:53:22.906415 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:53:22.906871 (+ 456us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:53:22.906875 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:53:22.906918 (+ 43us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:53:22.907112 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:53:22.907125 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:53:22.908376 (+ 1251us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:53:22.908381 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:53:22.909918 (+ 1537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:53:22.909930 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:53:22.911242 (+ 1312us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:53:22.911248 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:53:22.911428 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:22.911462 (+ 34us) client_negotiation.cc:769] Sending connection context
0809 22:53:22.911548 (+ 86us) client_negotiation.cc:240] Negotiation successful
0809 22:53:22.911690 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":215,"mutex_wait_us":39}
I20260809 22:53:22.912051 28892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:22.906024 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.113.131:45401 (local address 127.15.113.129:42763)
0809 22:53:22.906296 (+ 272us) server_negotiation.cc:206] Beginning negotiation
0809 22:53:22.906303 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:53:22.906325 (+ 22us) server_negotiation.cc:407] Connection header received
0809 22:53:22.906634 (+ 309us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:53:22.906640 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:53:22.906732 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:53:22.906832 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:53:22.907265 (+ 433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:22.908226 (+ 961us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:22.910085 (+ 1859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:22.911095 (+ 1010us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:22.911223 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:22.911280 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0809 22:53:22.911589 (+ 309us) server_negotiation.cc:299] Negotiation successful
0809 22:53:22.911723 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":97}
I20260809 22:53:22.912533 28912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:22.906012 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.130:46333 (local address 127.15.113.131:60383)
0809 22:53:22.906653 (+ 641us) negotiation.cc:107] Waiting for socket to connect
0809 22:53:22.906671 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 22:53:22.906802 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:53:22.907135 (+ 333us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:53:22.907138 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:53:22.907160 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:53:22.907590 (+ 430us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:53:22.907605 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:53:22.908742 (+ 1137us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:53:22.908748 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:53:22.910414 (+ 1666us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:53:22.910431 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:53:22.911650 (+ 1219us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:53:22.911658 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:53:22.911843 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:22.911880 (+ 37us) client_negotiation.cc:769] Sending connection context
0809 22:53:22.911993 (+ 113us) client_negotiation.cc:240] Negotiation successful
0809 22:53:22.912123 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":511,"mutex_wait_us":30,"thread_start_us":461,"threads_started":1}
I20260809 22:53:22.912570 28898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:22.906433 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.113.131:60383 (local address 127.15.113.130:46333)
0809 22:53:22.906614 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0809 22:53:22.906619 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:53:22.906750 (+ 131us) server_negotiation.cc:407] Connection header received
0809 22:53:22.906926 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:53:22.906930 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:53:22.907001 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:53:22.907101 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:53:22.907746 (+ 645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:22.908573 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:22.910581 (+ 2008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:22.911420 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:22.911574 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:22.911641 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0809 22:53:22.912296 (+ 655us) server_negotiation.cc:299] Negotiation successful
0809 22:53:22.912389 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260809 22:53:22.913166 28366 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5450590aa15f4d8998b0eba02a00f411" candidate_uuid: "ef758642c9de4d939a89e3d634d6a636" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b84b90efbbdb40bebe1b5adb43e9aa69" is_pre_election: true
I20260809 22:53:22.913578 28597 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5450590aa15f4d8998b0eba02a00f411" candidate_uuid: "ef758642c9de4d939a89e3d634d6a636" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a85606f73667466f804f9495aab892b4" is_pre_election: true
I20260809 22:53:22.913702 28366 raft_consensus.cc:2454] T 5450590aa15f4d8998b0eba02a00f411 P b84b90efbbdb40bebe1b5adb43e9aa69 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef758642c9de4d939a89e3d634d6a636 in term 0.
I20260809 22:53:22.914101 28597 raft_consensus.cc:2454] T 5450590aa15f4d8998b0eba02a00f411 P a85606f73667466f804f9495aab892b4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef758642c9de4d939a89e3d634d6a636 in term 0.
I20260809 22:53:22.914316 28690 leader_election.cc:302] T 5450590aa15f4d8998b0eba02a00f411 P ef758642c9de4d939a89e3d634d6a636 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b84b90efbbdb40bebe1b5adb43e9aa69, ef758642c9de4d939a89e3d634d6a636; no voters:
I20260809 22:53:22.914687 28908 raft_consensus.cc:2781] T 5450590aa15f4d8998b0eba02a00f411 P ef758642c9de4d939a89e3d634d6a636 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 22:53:22.914835 28908 raft_consensus.cc:482] T 5450590aa15f4d8998b0eba02a00f411 P ef758642c9de4d939a89e3d634d6a636 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 22:53:22.914934 28908 raft_consensus.cc:3037] T 5450590aa15f4d8998b0eba02a00f411 P ef758642c9de4d939a89e3d634d6a636 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:53:22.917047 28908 raft_consensus.cc:504] T 5450590aa15f4d8998b0eba02a00f411 P ef758642c9de4d939a89e3d634d6a636 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b84b90efbbdb40bebe1b5adb43e9aa69" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 42763 } } peers { permanent_uuid: "a85606f73667466f804f9495aab892b4" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 46333 } } peers { permanent_uuid: "ef758642c9de4d939a89e3d634d6a636" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 41323 } }
I20260809 22:53:22.917591 28908 leader_election.cc:288] T 5450590aa15f4d8998b0eba02a00f411 P ef758642c9de4d939a89e3d634d6a636 [CANDIDATE]: Term 1 election: Requested vote from peers b84b90efbbdb40bebe1b5adb43e9aa69 (127.15.113.129:42763), a85606f73667466f804f9495aab892b4 (127.15.113.130:46333)
I20260809 22:53:22.917954 28366 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5450590aa15f4d8998b0eba02a00f411" candidate_uuid: "ef758642c9de4d939a89e3d634d6a636" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b84b90efbbdb40bebe1b5adb43e9aa69"
I20260809 22:53:22.918084 28597 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5450590aa15f4d8998b0eba02a00f411" candidate_uuid: "ef758642c9de4d939a89e3d634d6a636" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a85606f73667466f804f9495aab892b4"
I20260809 22:53:22.918155 28366 raft_consensus.cc:3037] T 5450590aa15f4d8998b0eba02a00f411 P b84b90efbbdb40bebe1b5adb43e9aa69 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:53:22.918280 28597 raft_consensus.cc:3037] T 5450590aa15f4d8998b0eba02a00f411 P a85606f73667466f804f9495aab892b4 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:53:22.921315 28597 raft_consensus.cc:2454] T 5450590aa15f4d8998b0eba02a00f411 P a85606f73667466f804f9495aab892b4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef758642c9de4d939a89e3d634d6a636 in term 1.
I20260809 22:53:22.921309 28366 raft_consensus.cc:2454] T 5450590aa15f4d8998b0eba02a00f411 P b84b90efbbdb40bebe1b5adb43e9aa69 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef758642c9de4d939a89e3d634d6a636 in term 1.
I20260809 22:53:22.921713 28691 leader_election.cc:302] T 5450590aa15f4d8998b0eba02a00f411 P ef758642c9de4d939a89e3d634d6a636 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a85606f73667466f804f9495aab892b4, ef758642c9de4d939a89e3d634d6a636; no voters:
I20260809 22:53:22.922081 28908 raft_consensus.cc:2781] T 5450590aa15f4d8998b0eba02a00f411 P ef758642c9de4d939a89e3d634d6a636 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 22:53:22.922528 28908 raft_consensus.cc:686] T 5450590aa15f4d8998b0eba02a00f411 P ef758642c9de4d939a89e3d634d6a636 [term 1 LEADER]: Becoming Leader. State: Replica: ef758642c9de4d939a89e3d634d6a636, State: Running, Role: LEADER
I20260809 22:53:22.922987 28908 consensus_queue.cc:237] T 5450590aa15f4d8998b0eba02a00f411 P ef758642c9de4d939a89e3d634d6a636 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b84b90efbbdb40bebe1b5adb43e9aa69" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 42763 } } peers { permanent_uuid: "a85606f73667466f804f9495aab892b4" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 46333 } } peers { permanent_uuid: "ef758642c9de4d939a89e3d634d6a636" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 41323 } }
I20260809 22:53:22.927263 28128 catalog_manager.cc:5495] T 5450590aa15f4d8998b0eba02a00f411 P ef758642c9de4d939a89e3d634d6a636 reported cstate change: term changed from 0 to 1, leader changed from <none> to ef758642c9de4d939a89e3d634d6a636 (127.15.113.131). New cstate: current_term: 1 leader_uuid: "ef758642c9de4d939a89e3d634d6a636" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b84b90efbbdb40bebe1b5adb43e9aa69" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 42763 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a85606f73667466f804f9495aab892b4" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 46333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ef758642c9de4d939a89e3d634d6a636" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 41323 } health_report { overall_health: HEALTHY } } }
I20260809 22:53:22.962898 28899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:22.957115 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49770 (local address 127.15.113.131:41323)
0809 22:53:22.957303 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0809 22:53:22.957309 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:53:22.957485 (+ 176us) server_negotiation.cc:407] Connection header received
0809 22:53:22.957680 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:53:22.957685 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:53:22.957777 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:53:22.957912 (+ 135us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 22:53:22.958193 (+ 281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:22.959261 (+ 1068us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:22.960517 (+ 1256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:22.961177 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:22.961293 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:22.961680 (+ 387us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 22:53:22.962349 (+ 669us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 22:53:22.962490 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0809 22:53:22.962541 (+ 51us) server_negotiation.cc:299] Negotiation successful
0809 22:53:22.962634 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260809 22:53:22.970526 28366 raft_consensus.cc:1261] T 5450590aa15f4d8998b0eba02a00f411 P b84b90efbbdb40bebe1b5adb43e9aa69 [term 1 FOLLOWER]: Refusing update from remote peer ef758642c9de4d939a89e3d634d6a636: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 22:53:22.970528 28597 raft_consensus.cc:1261] T 5450590aa15f4d8998b0eba02a00f411 P a85606f73667466f804f9495aab892b4 [term 1 FOLLOWER]: Refusing update from remote peer ef758642c9de4d939a89e3d634d6a636: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 22:53:22.971458 28908 consensus_queue.cc:1035] T 5450590aa15f4d8998b0eba02a00f411 P ef758642c9de4d939a89e3d634d6a636 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a85606f73667466f804f9495aab892b4" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 46333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 22:53:22.971751 28914 consensus_queue.cc:1035] T 5450590aa15f4d8998b0eba02a00f411 P ef758642c9de4d939a89e3d634d6a636 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b84b90efbbdb40bebe1b5adb43e9aa69" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 42763 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 22:53:22.982875 28920 mvcc.cc:205] Tried to move back new op lower bound from 7316750348158025728 to 7316750347976384512. Current Snapshot: MvccSnapshot[applied={T|T < 7316750348158025728}]
I20260809 22:53:22.994720 28128 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:49134:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 22:53:22.994954 28128 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:49134:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 22:53:22.998337 28128 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f37bab48ada649e6bd170b082f14874f: Sending DeleteTablet for 3 replicas of tablet 5450590aa15f4d8998b0eba02a00f411
I20260809 22:53:22.999414 28330 tablet_service.cc:1514] Processing DeleteTablet for tablet 5450590aa15f4d8998b0eba02a00f411 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:53:22 UTC) from {username='oryx', principal='oryx/127.15.113.190@KRBTEST.COM'} at 127.0.0.1:35596
I20260809 22:53:22.999442 28549 tablet_service.cc:1514] Processing DeleteTablet for tablet 5450590aa15f4d8998b0eba02a00f411 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:53:22 UTC) from {username='oryx', principal='oryx/127.15.113.190@KRBTEST.COM'} at 127.0.0.1:37298
I20260809 22:53:23.000337 28765 tablet_service.cc:1514] Processing DeleteTablet for tablet 5450590aa15f4d8998b0eba02a00f411 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:53:22 UTC) from {username='oryx', principal='oryx/127.15.113.190@KRBTEST.COM'} at 127.0.0.1:49762
I20260809 22:53:23.001127 28940 tablet_replica.cc:330] T 5450590aa15f4d8998b0eba02a00f411 P ef758642c9de4d939a89e3d634d6a636: stopping tablet replica
I20260809 22:53:23.001155 28127 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:49134:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 22:53:23.001776 28127 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 22:53:23.001739 28940 raft_consensus.cc:2229] T 5450590aa15f4d8998b0eba02a00f411 P ef758642c9de4d939a89e3d634d6a636 [term 1 LEADER]: Raft consensus shutting down.
I20260809 22:53:23.002538 28940 raft_consensus.cc:2258] T 5450590aa15f4d8998b0eba02a00f411 P ef758642c9de4d939a89e3d634d6a636 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 22:53:23.008723 28328 tablet_service.cc:1467] Processing CreateTablet for tablet 9c6278eaa9d5483e9757cc1755a11122 (DEFAULT_TABLE table=test-table [id=6cfc3601a2b44b029db05ee339f2e62b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:53:23.009967 28548 tablet_service.cc:1467] Processing CreateTablet for tablet 9c6278eaa9d5483e9757cc1755a11122 (DEFAULT_TABLE table=test-table [id=6cfc3601a2b44b029db05ee339f2e62b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:53:23.010465 28548 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9c6278eaa9d5483e9757cc1755a11122. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:53:23.013252 28328 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9c6278eaa9d5483e9757cc1755a11122. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:53:23.017257 28765 tablet_service.cc:1467] Processing CreateTablet for tablet 9c6278eaa9d5483e9757cc1755a11122 (DEFAULT_TABLE table=test-table [id=6cfc3601a2b44b029db05ee339f2e62b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:53:23.017690 28765 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9c6278eaa9d5483e9757cc1755a11122. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:53:23.020531 28901 tablet_bootstrap.cc:492] T 9c6278eaa9d5483e9757cc1755a11122 P a85606f73667466f804f9495aab892b4: Bootstrap starting.
I20260809 22:53:23.022207 28901 tablet_bootstrap.cc:654] T 9c6278eaa9d5483e9757cc1755a11122 P a85606f73667466f804f9495aab892b4: Neither blocks nor log segments found. Creating new log.
I20260809 22:53:23.026432 28940 ts_tablet_manager.cc:1905] T 5450590aa15f4d8998b0eba02a00f411 P ef758642c9de4d939a89e3d634d6a636: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 22:53:23.026567 28902 tablet_bootstrap.cc:492] T 9c6278eaa9d5483e9757cc1755a11122 P b84b90efbbdb40bebe1b5adb43e9aa69: Bootstrap starting.
I20260809 22:53:23.028265 28902 tablet_bootstrap.cc:654] T 9c6278eaa9d5483e9757cc1755a11122 P b84b90efbbdb40bebe1b5adb43e9aa69: Neither blocks nor log segments found. Creating new log.
I20260809 22:53:23.034103 28901 tablet_bootstrap.cc:492] T 9c6278eaa9d5483e9757cc1755a11122 P a85606f73667466f804f9495aab892b4: No bootstrap required, opened a new log
I20260809 22:53:23.034278 28901 ts_tablet_manager.cc:1397] T 9c6278eaa9d5483e9757cc1755a11122 P a85606f73667466f804f9495aab892b4: Time spent bootstrapping tablet: real 0.014s user 0.009s sys 0.000s
I20260809 22:53:23.035272 28900 tablet_bootstrap.cc:492] T 9c6278eaa9d5483e9757cc1755a11122 P ef758642c9de4d939a89e3d634d6a636: Bootstrap starting.
I20260809 22:53:23.035159 28901 raft_consensus.cc:348] T 9c6278eaa9d5483e9757cc1755a11122 P a85606f73667466f804f9495aab892b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b84b90efbbdb40bebe1b5adb43e9aa69" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 42763 } } peers { permanent_uuid: "a85606f73667466f804f9495aab892b4" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 46333 } } peers { permanent_uuid: "ef758642c9de4d939a89e3d634d6a636" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 41323 } }
I20260809 22:53:23.035418 28901 raft_consensus.cc:374] T 9c6278eaa9d5483e9757cc1755a11122 P a85606f73667466f804f9495aab892b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:53:23.035526 28901 raft_consensus.cc:729] T 9c6278eaa9d5483e9757cc1755a11122 P a85606f73667466f804f9495aab892b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a85606f73667466f804f9495aab892b4, State: Initialized, Role: FOLLOWER
I20260809 22:53:23.035323 28938 tablet_replica.cc:330] T 5450590aa15f4d8998b0eba02a00f411 P b84b90efbbdb40bebe1b5adb43e9aa69: stopping tablet replica
I20260809 22:53:23.035889 28901 consensus_queue.cc:260] T 9c6278eaa9d5483e9757cc1755a11122 P a85606f73667466f804f9495aab892b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b84b90efbbdb40bebe1b5adb43e9aa69" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 42763 } } peers { permanent_uuid: "a85606f73667466f804f9495aab892b4" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 46333 } } peers { permanent_uuid: "ef758642c9de4d939a89e3d634d6a636" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 41323 } }
I20260809 22:53:23.036041 28938 raft_consensus.cc:2229] T 5450590aa15f4d8998b0eba02a00f411 P b84b90efbbdb40bebe1b5adb43e9aa69 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 22:53:23.036466 28901 ts_tablet_manager.cc:1428] T 9c6278eaa9d5483e9757cc1755a11122 P a85606f73667466f804f9495aab892b4: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260809 22:53:23.036706 28938 raft_consensus.cc:2258] T 5450590aa15f4d8998b0eba02a00f411 P b84b90efbbdb40bebe1b5adb43e9aa69 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 22:53:23.036777 28900 tablet_bootstrap.cc:654] T 9c6278eaa9d5483e9757cc1755a11122 P ef758642c9de4d939a89e3d634d6a636: Neither blocks nor log segments found. Creating new log.
I20260809 22:53:23.039765 28940 ts_tablet_manager.cc:1918] T 5450590aa15f4d8998b0eba02a00f411 P ef758642c9de4d939a89e3d634d6a636: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 22:53:23.040300 28940 log.cc:1196] T 5450590aa15f4d8998b0eba02a00f411 P ef758642c9de4d939a89e3d634d6a636: Deleting WAL directory at /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315935617919-15814-0/minicluster-data/ts-2/wal/wals/5450590aa15f4d8998b0eba02a00f411
I20260809 22:53:23.040946 28940 ts_tablet_manager.cc:1939] T 5450590aa15f4d8998b0eba02a00f411 P ef758642c9de4d939a89e3d634d6a636: Deleting consensus metadata
I20260809 22:53:23.041513 28937 tablet_replica.cc:330] T 5450590aa15f4d8998b0eba02a00f411 P a85606f73667466f804f9495aab892b4: stopping tablet replica
I20260809 22:53:23.041992 28937 raft_consensus.cc:2229] T 5450590aa15f4d8998b0eba02a00f411 P a85606f73667466f804f9495aab892b4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 22:53:23.043190 28937 raft_consensus.cc:2258] T 5450590aa15f4d8998b0eba02a00f411 P a85606f73667466f804f9495aab892b4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 22:53:23.044006 28106 catalog_manager.cc:4841] TS ef758642c9de4d939a89e3d634d6a636 (127.15.113.131:41323): tablet 5450590aa15f4d8998b0eba02a00f411 (table test-table [id=4310a815351a4642a007c824068f2334]) successfully deleted
I20260809 22:53:23.045298 28938 ts_tablet_manager.cc:1905] T 5450590aa15f4d8998b0eba02a00f411 P b84b90efbbdb40bebe1b5adb43e9aa69: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 22:53:23.046136 28937 ts_tablet_manager.cc:1905] T 5450590aa15f4d8998b0eba02a00f411 P a85606f73667466f804f9495aab892b4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 22:53:23.052067 28900 tablet_bootstrap.cc:492] T 9c6278eaa9d5483e9757cc1755a11122 P ef758642c9de4d939a89e3d634d6a636: No bootstrap required, opened a new log
I20260809 22:53:23.052227 28900 ts_tablet_manager.cc:1397] T 9c6278eaa9d5483e9757cc1755a11122 P ef758642c9de4d939a89e3d634d6a636: Time spent bootstrapping tablet: real 0.017s user 0.004s sys 0.003s
I20260809 22:53:23.053038 28900 raft_consensus.cc:348] T 9c6278eaa9d5483e9757cc1755a11122 P ef758642c9de4d939a89e3d634d6a636 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b84b90efbbdb40bebe1b5adb43e9aa69" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 42763 } } peers { permanent_uuid: "a85606f73667466f804f9495aab892b4" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 46333 } } peers { permanent_uuid: "ef758642c9de4d939a89e3d634d6a636" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 41323 } }
I20260809 22:53:23.053268 28900 raft_consensus.cc:374] T 9c6278eaa9d5483e9757cc1755a11122 P ef758642c9de4d939a89e3d634d6a636 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:53:23.053376 28900 raft_consensus.cc:729] T 9c6278eaa9d5483e9757cc1755a11122 P ef758642c9de4d939a89e3d634d6a636 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef758642c9de4d939a89e3d634d6a636, State: Initialized, Role: FOLLOWER
I20260809 22:53:23.053504 28938 ts_tablet_manager.cc:1918] T 5450590aa15f4d8998b0eba02a00f411 P b84b90efbbdb40bebe1b5adb43e9aa69: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 22:53:23.053670 28938 log.cc:1196] T 5450590aa15f4d8998b0eba02a00f411 P b84b90efbbdb40bebe1b5adb43e9aa69: Deleting WAL directory at /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315935617919-15814-0/minicluster-data/ts-0/wal/wals/5450590aa15f4d8998b0eba02a00f411
I20260809 22:53:23.053722 28900 consensus_queue.cc:260] T 9c6278eaa9d5483e9757cc1755a11122 P ef758642c9de4d939a89e3d634d6a636 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b84b90efbbdb40bebe1b5adb43e9aa69" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 42763 } } peers { permanent_uuid: "a85606f73667466f804f9495aab892b4" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 46333 } } peers { permanent_uuid: "ef758642c9de4d939a89e3d634d6a636" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 41323 } }
I20260809 22:53:23.053860 28937 ts_tablet_manager.cc:1918] T 5450590aa15f4d8998b0eba02a00f411 P a85606f73667466f804f9495aab892b4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 22:53:23.054117 28937 log.cc:1196] T 5450590aa15f4d8998b0eba02a00f411 P a85606f73667466f804f9495aab892b4: Deleting WAL directory at /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315935617919-15814-0/minicluster-data/ts-1/wal/wals/5450590aa15f4d8998b0eba02a00f411
I20260809 22:53:23.054224 28938 ts_tablet_manager.cc:1939] T 5450590aa15f4d8998b0eba02a00f411 P b84b90efbbdb40bebe1b5adb43e9aa69: Deleting consensus metadata
I20260809 22:53:23.054759 28937 ts_tablet_manager.cc:1939] T 5450590aa15f4d8998b0eba02a00f411 P a85606f73667466f804f9495aab892b4: Deleting consensus metadata
I20260809 22:53:23.056195 28102 catalog_manager.cc:4841] TS b84b90efbbdb40bebe1b5adb43e9aa69 (127.15.113.129:42763): tablet 5450590aa15f4d8998b0eba02a00f411 (table test-table [id=4310a815351a4642a007c824068f2334]) successfully deleted
I20260809 22:53:23.056399 28900 ts_tablet_manager.cc:1428] T 9c6278eaa9d5483e9757cc1755a11122 P ef758642c9de4d939a89e3d634d6a636: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260809 22:53:23.056470 28103 catalog_manager.cc:4841] TS a85606f73667466f804f9495aab892b4 (127.15.113.130:46333): tablet 5450590aa15f4d8998b0eba02a00f411 (table test-table [id=4310a815351a4642a007c824068f2334]) successfully deleted
I20260809 22:53:23.057672 28902 tablet_bootstrap.cc:492] T 9c6278eaa9d5483e9757cc1755a11122 P b84b90efbbdb40bebe1b5adb43e9aa69: No bootstrap required, opened a new log
I20260809 22:53:23.057806 28902 ts_tablet_manager.cc:1397] T 9c6278eaa9d5483e9757cc1755a11122 P b84b90efbbdb40bebe1b5adb43e9aa69: Time spent bootstrapping tablet: real 0.031s user 0.005s sys 0.004s
I20260809 22:53:23.058445 28902 raft_consensus.cc:348] T 9c6278eaa9d5483e9757cc1755a11122 P b84b90efbbdb40bebe1b5adb43e9aa69 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b84b90efbbdb40bebe1b5adb43e9aa69" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 42763 } } peers { permanent_uuid: "a85606f73667466f804f9495aab892b4" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 46333 } } peers { permanent_uuid: "ef758642c9de4d939a89e3d634d6a636" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 41323 } }
I20260809 22:53:23.058666 28902 raft_consensus.cc:374] T 9c6278eaa9d5483e9757cc1755a11122 P b84b90efbbdb40bebe1b5adb43e9aa69 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:53:23.058766 28902 raft_consensus.cc:729] T 9c6278eaa9d5483e9757cc1755a11122 P b84b90efbbdb40bebe1b5adb43e9aa69 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b84b90efbbdb40bebe1b5adb43e9aa69, State: Initialized, Role: FOLLOWER
I20260809 22:53:23.058990 28902 consensus_queue.cc:260] T 9c6278eaa9d5483e9757cc1755a11122 P b84b90efbbdb40bebe1b5adb43e9aa69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b84b90efbbdb40bebe1b5adb43e9aa69" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 42763 } } peers { permanent_uuid: "a85606f73667466f804f9495aab892b4" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 46333 } } peers { permanent_uuid: "ef758642c9de4d939a89e3d634d6a636" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 41323 } }
I20260809 22:53:23.059433 28902 ts_tablet_manager.cc:1428] T 9c6278eaa9d5483e9757cc1755a11122 P b84b90efbbdb40bebe1b5adb43e9aa69: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260809 22:53:23.289366 28914 raft_consensus.cc:482] T 9c6278eaa9d5483e9757cc1755a11122 P ef758642c9de4d939a89e3d634d6a636 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 22:53:23.289592 28914 raft_consensus.cc:504] T 9c6278eaa9d5483e9757cc1755a11122 P ef758642c9de4d939a89e3d634d6a636 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b84b90efbbdb40bebe1b5adb43e9aa69" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 42763 } } peers { permanent_uuid: "a85606f73667466f804f9495aab892b4" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 46333 } } peers { permanent_uuid: "ef758642c9de4d939a89e3d634d6a636" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 41323 } }
I20260809 22:53:23.290269 28914 leader_election.cc:288] T 9c6278eaa9d5483e9757cc1755a11122 P ef758642c9de4d939a89e3d634d6a636 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b84b90efbbdb40bebe1b5adb43e9aa69 (127.15.113.129:42763), a85606f73667466f804f9495aab892b4 (127.15.113.130:46333)
I20260809 22:53:23.290920 28597 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9c6278eaa9d5483e9757cc1755a11122" candidate_uuid: "ef758642c9de4d939a89e3d634d6a636" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a85606f73667466f804f9495aab892b4" is_pre_election: true
I20260809 22:53:23.290953 28366 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9c6278eaa9d5483e9757cc1755a11122" candidate_uuid: "ef758642c9de4d939a89e3d634d6a636" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b84b90efbbdb40bebe1b5adb43e9aa69" is_pre_election: true
I20260809 22:53:23.291200 28366 raft_consensus.cc:2454] T 9c6278eaa9d5483e9757cc1755a11122 P b84b90efbbdb40bebe1b5adb43e9aa69 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef758642c9de4d939a89e3d634d6a636 in term 0.
I20260809 22:53:23.291201 28597 raft_consensus.cc:2454] T 9c6278eaa9d5483e9757cc1755a11122 P a85606f73667466f804f9495aab892b4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef758642c9de4d939a89e3d634d6a636 in term 0.
I20260809 22:53:23.291656 28691 leader_election.cc:302] T 9c6278eaa9d5483e9757cc1755a11122 P ef758642c9de4d939a89e3d634d6a636 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a85606f73667466f804f9495aab892b4, ef758642c9de4d939a89e3d634d6a636; no voters:
I20260809 22:53:23.291968 28914 raft_consensus.cc:2781] T 9c6278eaa9d5483e9757cc1755a11122 P ef758642c9de4d939a89e3d634d6a636 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 22:53:23.292104 28914 raft_consensus.cc:482] T 9c6278eaa9d5483e9757cc1755a11122 P ef758642c9de4d939a89e3d634d6a636 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 22:53:23.292236 28914 raft_consensus.cc:3037] T 9c6278eaa9d5483e9757cc1755a11122 P ef758642c9de4d939a89e3d634d6a636 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:53:23.295598 28914 raft_consensus.cc:504] T 9c6278eaa9d5483e9757cc1755a11122 P ef758642c9de4d939a89e3d634d6a636 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b84b90efbbdb40bebe1b5adb43e9aa69" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 42763 } } peers { permanent_uuid: "a85606f73667466f804f9495aab892b4" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 46333 } } peers { permanent_uuid: "ef758642c9de4d939a89e3d634d6a636" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 41323 } }
I20260809 22:53:23.296303 28914 leader_election.cc:288] T 9c6278eaa9d5483e9757cc1755a11122 P ef758642c9de4d939a89e3d634d6a636 [CANDIDATE]: Term 1 election: Requested vote from peers b84b90efbbdb40bebe1b5adb43e9aa69 (127.15.113.129:42763), a85606f73667466f804f9495aab892b4 (127.15.113.130:46333)
I20260809 22:53:23.296700 28366 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9c6278eaa9d5483e9757cc1755a11122" candidate_uuid: "ef758642c9de4d939a89e3d634d6a636" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b84b90efbbdb40bebe1b5adb43e9aa69"
I20260809 22:53:23.296882 28366 raft_consensus.cc:3037] T 9c6278eaa9d5483e9757cc1755a11122 P b84b90efbbdb40bebe1b5adb43e9aa69 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:53:23.296797 28597 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9c6278eaa9d5483e9757cc1755a11122" candidate_uuid: "ef758642c9de4d939a89e3d634d6a636" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a85606f73667466f804f9495aab892b4"
I20260809 22:53:23.297049 28597 raft_consensus.cc:3037] T 9c6278eaa9d5483e9757cc1755a11122 P a85606f73667466f804f9495aab892b4 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:53:23.298992 28366 raft_consensus.cc:2454] T 9c6278eaa9d5483e9757cc1755a11122 P b84b90efbbdb40bebe1b5adb43e9aa69 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef758642c9de4d939a89e3d634d6a636 in term 1.
I20260809 22:53:23.299371 28690 leader_election.cc:302] T 9c6278eaa9d5483e9757cc1755a11122 P ef758642c9de4d939a89e3d634d6a636 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b84b90efbbdb40bebe1b5adb43e9aa69, ef758642c9de4d939a89e3d634d6a636; no voters:
I20260809 22:53:23.299644 28914 raft_consensus.cc:2781] T 9c6278eaa9d5483e9757cc1755a11122 P ef758642c9de4d939a89e3d634d6a636 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 22:53:23.299795 28914 raft_consensus.cc:686] T 9c6278eaa9d5483e9757cc1755a11122 P ef758642c9de4d939a89e3d634d6a636 [term 1 LEADER]: Becoming Leader. State: Replica: ef758642c9de4d939a89e3d634d6a636, State: Running, Role: LEADER
I20260809 22:53:23.300117 28597 raft_consensus.cc:2454] T 9c6278eaa9d5483e9757cc1755a11122 P a85606f73667466f804f9495aab892b4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef758642c9de4d939a89e3d634d6a636 in term 1.
I20260809 22:53:23.300431 28914 consensus_queue.cc:237] T 9c6278eaa9d5483e9757cc1755a11122 P ef758642c9de4d939a89e3d634d6a636 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b84b90efbbdb40bebe1b5adb43e9aa69" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 42763 } } peers { permanent_uuid: "a85606f73667466f804f9495aab892b4" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 46333 } } peers { permanent_uuid: "ef758642c9de4d939a89e3d634d6a636" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 41323 } }
I20260809 22:53:23.303544 28128 catalog_manager.cc:5495] T 9c6278eaa9d5483e9757cc1755a11122 P ef758642c9de4d939a89e3d634d6a636 reported cstate change: term changed from 0 to 1, leader changed from <none> to ef758642c9de4d939a89e3d634d6a636 (127.15.113.131). New cstate: current_term: 1 leader_uuid: "ef758642c9de4d939a89e3d634d6a636" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b84b90efbbdb40bebe1b5adb43e9aa69" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 42763 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a85606f73667466f804f9495aab892b4" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 46333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ef758642c9de4d939a89e3d634d6a636" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 41323 } health_report { overall_health: HEALTHY } } }
Aug 09 22:53:23 dist-test-slave-ncp9 krb5kdc[28066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316000, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.113.190@KRBTEST.COM for oryx/127.15.113.190@KRBTEST.COM
I20260809 22:53:23.349330 28954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:23.331235 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49148 (local address 127.15.113.190:34881)
0809 22:53:23.331841 (+ 606us) server_negotiation.cc:206] Beginning negotiation
0809 22:53:23.331848 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:53:23.332159 (+ 311us) server_negotiation.cc:407] Connection header received
0809 22:53:23.332323 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:53:23.332328 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:53:23.332423 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:53:23.332552 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:53:23.333597 (+ 1045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:23.334778 (+ 1181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:23.336093 (+ 1315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:23.336766 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:23.336935 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:23.343930 (+ 6995us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:53:23.343935 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:53:23.343938 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:53:23.343967 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:53:23.346533 (+ 2566us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:53:23.347427 (+ 894us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:53:23.347430 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:53:23.347432 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:53:23.347548 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:53:23.347967 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:53:23.347970 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:53:23.347972 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:53:23.348226 (+ 254us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:53:23.348337 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0809 22:53:23.348842 (+ 505us) server_negotiation.cc:299] Negotiation successful
0809 22:53:23.349040 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":433,"thread_start_us":269,"threads_started":1}
I20260809 22:53:23.349720 28953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:23.331155 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:34881 (local address 127.0.0.1:49148)
0809 22:53:23.332047 (+ 892us) negotiation.cc:107] Waiting for socket to connect
0809 22:53:23.332063 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 22:53:23.332208 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:53:23.332582 (+ 374us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:53:23.332585 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:53:23.333263 (+ 678us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:53:23.333444 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:53:23.333452 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:53:23.334912 (+ 1460us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:53:23.334916 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:53:23.335941 (+ 1025us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:53:23.335954 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:53:23.336953 (+ 999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:53:23.336958 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:53:23.337077 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:23.341210 (+ 4133us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:53:23.341227 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:53:23.343734 (+ 2507us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:53:23.346666 (+ 2932us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:53:23.346681 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:53:23.346690 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:53:23.347317 (+ 627us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:53:23.347662 (+ 345us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:53:23.347668 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:53:23.347671 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:53:23.347862 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:53:23.348363 (+ 501us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:53:23.348372 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:53:23.348581 (+ 209us) client_negotiation.cc:769] Sending connection context
0809 22:53:23.348787 (+ 206us) client_negotiation.cc:240] Negotiation successful
0809 22:53:23.349233 (+ 446us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":763,"mutex_wait_us":98,"thread_start_us":606,"threads_started":1}
I20260809 22:53:23.355195 28128 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.15.113.190@KRBTEST.COM'} at 127.0.0.1:49148:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260809 22:53:23.356180 28128 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 22:53:23.362787 28330 tablet_service.cc:1467] Processing CreateTablet for tablet 33861f2c19dc4d9d9ffe3a875a799fc7 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=f487e2102fad4be18f503dc204207bff]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 22:53:23.363195 28549 tablet_service.cc:1467] Processing CreateTablet for tablet 33861f2c19dc4d9d9ffe3a875a799fc7 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=f487e2102fad4be18f503dc204207bff]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 22:53:23.363394 28330 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 33861f2c19dc4d9d9ffe3a875a799fc7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:53:23.363655 28549 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 33861f2c19dc4d9d9ffe3a875a799fc7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:53:23.368409 28765 tablet_service.cc:1467] Processing CreateTablet for tablet 33861f2c19dc4d9d9ffe3a875a799fc7 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=f487e2102fad4be18f503dc204207bff]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 22:53:23.368992 28765 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 33861f2c19dc4d9d9ffe3a875a799fc7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:53:23.369356 28901 tablet_bootstrap.cc:492] T 33861f2c19dc4d9d9ffe3a875a799fc7 P a85606f73667466f804f9495aab892b4: Bootstrap starting.
I20260809 22:53:23.370975 28901 tablet_bootstrap.cc:654] T 33861f2c19dc4d9d9ffe3a875a799fc7 P a85606f73667466f804f9495aab892b4: Neither blocks nor log segments found. Creating new log.
I20260809 22:53:23.372968 28901 tablet_bootstrap.cc:492] T 33861f2c19dc4d9d9ffe3a875a799fc7 P a85606f73667466f804f9495aab892b4: No bootstrap required, opened a new log
I20260809 22:53:23.373127 28901 ts_tablet_manager.cc:1397] T 33861f2c19dc4d9d9ffe3a875a799fc7 P a85606f73667466f804f9495aab892b4: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260809 22:53:23.373816 28901 raft_consensus.cc:348] T 33861f2c19dc4d9d9ffe3a875a799fc7 P a85606f73667466f804f9495aab892b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b84b90efbbdb40bebe1b5adb43e9aa69" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 42763 } } peers { permanent_uuid: "ef758642c9de4d939a89e3d634d6a636" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 41323 } } peers { permanent_uuid: "a85606f73667466f804f9495aab892b4" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 46333 } }
I20260809 22:53:23.374037 28901 raft_consensus.cc:374] T 33861f2c19dc4d9d9ffe3a875a799fc7 P a85606f73667466f804f9495aab892b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:53:23.374097 28901 raft_consensus.cc:729] T 33861f2c19dc4d9d9ffe3a875a799fc7 P a85606f73667466f804f9495aab892b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a85606f73667466f804f9495aab892b4, State: Initialized, Role: FOLLOWER
I20260809 22:53:23.374249 28900 tablet_bootstrap.cc:492] T 33861f2c19dc4d9d9ffe3a875a799fc7 P ef758642c9de4d939a89e3d634d6a636: Bootstrap starting.
I20260809 22:53:23.374377 28901 consensus_queue.cc:260] T 33861f2c19dc4d9d9ffe3a875a799fc7 P a85606f73667466f804f9495aab892b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b84b90efbbdb40bebe1b5adb43e9aa69" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 42763 } } peers { permanent_uuid: "ef758642c9de4d939a89e3d634d6a636" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 41323 } } peers { permanent_uuid: "a85606f73667466f804f9495aab892b4" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 46333 } }
I20260809 22:53:23.374822 28909 tablet_replica.cc:439] T 33861f2c19dc4d9d9ffe3a875a799fc7 P a85606f73667466f804f9495aab892b4: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b84b90efbbdb40bebe1b5adb43e9aa69" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 42763 } } peers { permanent_uuid: "ef758642c9de4d939a89e3d634d6a636" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 41323 } } peers { permanent_uuid: "a85606f73667466f804f9495aab892b4" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 46333 } } }
I20260809 22:53:23.375152 28909 tablet_replica.cc:442] T 33861f2c19dc4d9d9ffe3a875a799fc7 P a85606f73667466f804f9495aab892b4: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 22:53:23.375892 28900 tablet_bootstrap.cc:654] T 33861f2c19dc4d9d9ffe3a875a799fc7 P ef758642c9de4d939a89e3d634d6a636: Neither blocks nor log segments found. Creating new log.
I20260809 22:53:23.376124 28901 ts_tablet_manager.cc:1428] T 33861f2c19dc4d9d9ffe3a875a799fc7 P a85606f73667466f804f9495aab892b4: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260809 22:53:23.377260 28902 tablet_bootstrap.cc:492] T 33861f2c19dc4d9d9ffe3a875a799fc7 P b84b90efbbdb40bebe1b5adb43e9aa69: Bootstrap starting.
I20260809 22:53:23.379035 28902 tablet_bootstrap.cc:654] T 33861f2c19dc4d9d9ffe3a875a799fc7 P b84b90efbbdb40bebe1b5adb43e9aa69: Neither blocks nor log segments found. Creating new log.
I20260809 22:53:23.381259 28902 tablet_bootstrap.cc:492] T 33861f2c19dc4d9d9ffe3a875a799fc7 P b84b90efbbdb40bebe1b5adb43e9aa69: No bootstrap required, opened a new log
I20260809 22:53:23.381424 28902 ts_tablet_manager.cc:1397] T 33861f2c19dc4d9d9ffe3a875a799fc7 P b84b90efbbdb40bebe1b5adb43e9aa69: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.001s
I20260809 22:53:23.382107 28902 raft_consensus.cc:348] T 33861f2c19dc4d9d9ffe3a875a799fc7 P b84b90efbbdb40bebe1b5adb43e9aa69 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b84b90efbbdb40bebe1b5adb43e9aa69" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 42763 } } peers { permanent_uuid: "ef758642c9de4d939a89e3d634d6a636" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 41323 } } peers { permanent_uuid: "a85606f73667466f804f9495aab892b4" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 46333 } }
I20260809 22:53:23.382445 28902 raft_consensus.cc:374] T 33861f2c19dc4d9d9ffe3a875a799fc7 P b84b90efbbdb40bebe1b5adb43e9aa69 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:53:23.382491 28900 tablet_bootstrap.cc:492] T 33861f2c19dc4d9d9ffe3a875a799fc7 P ef758642c9de4d939a89e3d634d6a636: No bootstrap required, opened a new log
I20260809 22:53:23.382534 28902 raft_consensus.cc:729] T 33861f2c19dc4d9d9ffe3a875a799fc7 P b84b90efbbdb40bebe1b5adb43e9aa69 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b84b90efbbdb40bebe1b5adb43e9aa69, State: Initialized, Role: FOLLOWER
I20260809 22:53:23.382658 28900 ts_tablet_manager.cc:1397] T 33861f2c19dc4d9d9ffe3a875a799fc7 P ef758642c9de4d939a89e3d634d6a636: Time spent bootstrapping tablet: real 0.008s user 0.004s sys 0.003s
I20260809 22:53:23.382809 28902 consensus_queue.cc:260] T 33861f2c19dc4d9d9ffe3a875a799fc7 P b84b90efbbdb40bebe1b5adb43e9aa69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b84b90efbbdb40bebe1b5adb43e9aa69" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 42763 } } peers { permanent_uuid: "ef758642c9de4d939a89e3d634d6a636" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 41323 } } peers { permanent_uuid: "a85606f73667466f804f9495aab892b4" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 46333 } }
I20260809 22:53:23.383294 28911 tablet_replica.cc:439] T 33861f2c19dc4d9d9ffe3a875a799fc7 P b84b90efbbdb40bebe1b5adb43e9aa69: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b84b90efbbdb40bebe1b5adb43e9aa69" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 42763 } } peers { permanent_uuid: "ef758642c9de4d939a89e3d634d6a636" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 41323 } } peers { permanent_uuid: "a85606f73667466f804f9495aab892b4" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 46333 } } }
I20260809 22:53:23.383406 28900 raft_consensus.cc:348] T 33861f2c19dc4d9d9ffe3a875a799fc7 P ef758642c9de4d939a89e3d634d6a636 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b84b90efbbdb40bebe1b5adb43e9aa69" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 42763 } } peers { permanent_uuid: "ef758642c9de4d939a89e3d634d6a636" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 41323 } } peers { permanent_uuid: "a85606f73667466f804f9495aab892b4" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 46333 } }
I20260809 22:53:23.383641 28911 tablet_replica.cc:442] T 33861f2c19dc4d9d9ffe3a875a799fc7 P b84b90efbbdb40bebe1b5adb43e9aa69: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 22:53:23.383646 28900 raft_consensus.cc:374] T 33861f2c19dc4d9d9ffe3a875a799fc7 P ef758642c9de4d939a89e3d634d6a636 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:53:23.383742 28900 raft_consensus.cc:729] T 33861f2c19dc4d9d9ffe3a875a799fc7 P ef758642c9de4d939a89e3d634d6a636 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef758642c9de4d939a89e3d634d6a636, State: Initialized, Role: FOLLOWER
I20260809 22:53:23.384014 28900 consensus_queue.cc:260] T 33861f2c19dc4d9d9ffe3a875a799fc7 P ef758642c9de4d939a89e3d634d6a636 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b84b90efbbdb40bebe1b5adb43e9aa69" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 42763 } } peers { permanent_uuid: "ef758642c9de4d939a89e3d634d6a636" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 41323 } } peers { permanent_uuid: "a85606f73667466f804f9495aab892b4" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 46333 } }
I20260809 22:53:23.384373 28902 ts_tablet_manager.cc:1428] T 33861f2c19dc4d9d9ffe3a875a799fc7 P b84b90efbbdb40bebe1b5adb43e9aa69: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260809 22:53:23.384488 28914 tablet_replica.cc:439] T 33861f2c19dc4d9d9ffe3a875a799fc7 P ef758642c9de4d939a89e3d634d6a636: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b84b90efbbdb40bebe1b5adb43e9aa69" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 42763 } } peers { permanent_uuid: "ef758642c9de4d939a89e3d634d6a636" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 41323 } } peers { permanent_uuid: "a85606f73667466f804f9495aab892b4" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 46333 } } }
I20260809 22:53:23.384794 28914 tablet_replica.cc:442] T 33861f2c19dc4d9d9ffe3a875a799fc7 P ef758642c9de4d939a89e3d634d6a636: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 22:53:23.386268 28900 ts_tablet_manager.cc:1428] T 33861f2c19dc4d9d9ffe3a875a799fc7 P ef758642c9de4d939a89e3d634d6a636: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20260809 22:53:23.563902 28909 raft_consensus.cc:482] T 33861f2c19dc4d9d9ffe3a875a799fc7 P a85606f73667466f804f9495aab892b4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 22:53:23.564117 28909 raft_consensus.cc:504] T 33861f2c19dc4d9d9ffe3a875a799fc7 P a85606f73667466f804f9495aab892b4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b84b90efbbdb40bebe1b5adb43e9aa69" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 42763 } } peers { permanent_uuid: "ef758642c9de4d939a89e3d634d6a636" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 41323 } } peers { permanent_uuid: "a85606f73667466f804f9495aab892b4" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 46333 } }
I20260809 22:53:23.565668 28909 leader_election.cc:288] T 33861f2c19dc4d9d9ffe3a875a799fc7 P a85606f73667466f804f9495aab892b4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b84b90efbbdb40bebe1b5adb43e9aa69 (127.15.113.129:42763), ef758642c9de4d939a89e3d634d6a636 (127.15.113.131:41323)
I20260809 22:53:23.583244 28962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:23.565956 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.113.130:46035 (local address 127.15.113.129:42763)
0809 22:53:23.577942 (+ 11986us) server_negotiation.cc:206] Beginning negotiation
0809 22:53:23.577947 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:53:23.578300 (+ 353us) server_negotiation.cc:407] Connection header received
0809 22:53:23.578499 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:53:23.578503 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:53:23.578565 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:53:23.578659 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:53:23.579135 (+ 476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:23.579856 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:23.581568 (+ 1712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:23.582345 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:23.582502 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:23.582577 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0809 22:53:23.582819 (+ 242us) server_negotiation.cc:299] Negotiation successful
0809 22:53:23.582959 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11853,"thread_start_us":11757,"threads_started":1}
I20260809 22:53:23.583333 28965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:23.566165 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.113.130:33447 (local address 127.15.113.131:41323)
0809 22:53:23.577759 (+ 11594us) server_negotiation.cc:206] Beginning negotiation
0809 22:53:23.577765 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:53:23.578382 (+ 617us) server_negotiation.cc:407] Connection header received
0809 22:53:23.578561 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:53:23.578565 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:53:23.578625 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:53:23.578706 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:53:23.579281 (+ 575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:23.579975 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:23.581666 (+ 1691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:23.582430 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:23.582580 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:23.582651 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0809 22:53:23.582915 (+ 264us) server_negotiation.cc:299] Negotiation successful
0809 22:53:23.583056 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11461,"thread_start_us":11362,"threads_started":1}
I20260809 22:53:23.583609 28961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:23.565863 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.129:42763 (local address 127.15.113.130:46035)
0809 22:53:23.578184 (+ 12321us) negotiation.cc:107] Waiting for socket to connect
0809 22:53:23.578204 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 22:53:23.578365 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:53:23.578677 (+ 312us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:53:23.578682 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:53:23.578706 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:53:23.578981 (+ 275us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:53:23.578991 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:53:23.579995 (+ 1004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:53:23.579999 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:53:23.581419 (+ 1420us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:53:23.581431 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:53:23.582503 (+ 1072us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:53:23.582509 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:53:23.582665 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:23.582689 (+ 24us) client_negotiation.cc:769] Sending connection context
0809 22:53:23.582774 (+ 85us) client_negotiation.cc:240] Negotiation successful
0809 22:53:23.582903 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12179,"thread_start_us":12077,"threads_started":1}
I20260809 22:53:23.583666 28963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:23.566017 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.131:41323 (local address 127.15.113.130:33447)
0809 22:53:23.578276 (+ 12259us) negotiation.cc:107] Waiting for socket to connect
0809 22:53:23.578290 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 22:53:23.578420 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:53:23.578737 (+ 317us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:53:23.578741 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:53:23.578783 (+ 42us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:53:23.579127 (+ 344us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:53:23.579136 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:53:23.580114 (+ 978us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:53:23.580121 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:53:23.581530 (+ 1409us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:53:23.581543 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:53:23.582620 (+ 1077us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:53:23.582626 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:53:23.582772 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:23.582796 (+ 24us) client_negotiation.cc:769] Sending connection context
0809 22:53:23.582872 (+ 76us) client_negotiation.cc:240] Negotiation successful
0809 22:53:23.582991 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12163,"thread_start_us":12033,"threads_started":1}
I20260809 22:53:23.584331 28366 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "33861f2c19dc4d9d9ffe3a875a799fc7" candidate_uuid: "a85606f73667466f804f9495aab892b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b84b90efbbdb40bebe1b5adb43e9aa69" is_pre_election: true
I20260809 22:53:23.584538 28804 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "33861f2c19dc4d9d9ffe3a875a799fc7" candidate_uuid: "a85606f73667466f804f9495aab892b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef758642c9de4d939a89e3d634d6a636" is_pre_election: true
I20260809 22:53:23.584664 28366 raft_consensus.cc:2454] T 33861f2c19dc4d9d9ffe3a875a799fc7 P b84b90efbbdb40bebe1b5adb43e9aa69 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a85606f73667466f804f9495aab892b4 in term 0.
I20260809 22:53:23.584950 28804 raft_consensus.cc:2454] T 33861f2c19dc4d9d9ffe3a875a799fc7 P ef758642c9de4d939a89e3d634d6a636 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a85606f73667466f804f9495aab892b4 in term 0.
I20260809 22:53:23.585160 28469 leader_election.cc:302] T 33861f2c19dc4d9d9ffe3a875a799fc7 P a85606f73667466f804f9495aab892b4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a85606f73667466f804f9495aab892b4, b84b90efbbdb40bebe1b5adb43e9aa69; no voters:
I20260809 22:53:23.585557 28909 raft_consensus.cc:2781] T 33861f2c19dc4d9d9ffe3a875a799fc7 P a85606f73667466f804f9495aab892b4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 22:53:23.585702 28909 raft_consensus.cc:482] T 33861f2c19dc4d9d9ffe3a875a799fc7 P a85606f73667466f804f9495aab892b4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 22:53:23.585798 28909 raft_consensus.cc:3037] T 33861f2c19dc4d9d9ffe3a875a799fc7 P a85606f73667466f804f9495aab892b4 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:53:23.587841 28909 raft_consensus.cc:504] T 33861f2c19dc4d9d9ffe3a875a799fc7 P a85606f73667466f804f9495aab892b4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b84b90efbbdb40bebe1b5adb43e9aa69" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 42763 } } peers { permanent_uuid: "ef758642c9de4d939a89e3d634d6a636" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 41323 } } peers { permanent_uuid: "a85606f73667466f804f9495aab892b4" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 46333 } }
I20260809 22:53:23.588403 28909 leader_election.cc:288] T 33861f2c19dc4d9d9ffe3a875a799fc7 P a85606f73667466f804f9495aab892b4 [CANDIDATE]: Term 1 election: Requested vote from peers b84b90efbbdb40bebe1b5adb43e9aa69 (127.15.113.129:42763), ef758642c9de4d939a89e3d634d6a636 (127.15.113.131:41323)
I20260809 22:53:23.588832 28804 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "33861f2c19dc4d9d9ffe3a875a799fc7" candidate_uuid: "a85606f73667466f804f9495aab892b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef758642c9de4d939a89e3d634d6a636"
I20260809 22:53:23.588821 28366 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "33861f2c19dc4d9d9ffe3a875a799fc7" candidate_uuid: "a85606f73667466f804f9495aab892b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b84b90efbbdb40bebe1b5adb43e9aa69"
I20260809 22:53:23.589047 28804 raft_consensus.cc:3037] T 33861f2c19dc4d9d9ffe3a875a799fc7 P ef758642c9de4d939a89e3d634d6a636 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:53:23.589084 28366 raft_consensus.cc:3037] T 33861f2c19dc4d9d9ffe3a875a799fc7 P b84b90efbbdb40bebe1b5adb43e9aa69 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:53:23.591871 28366 raft_consensus.cc:2454] T 33861f2c19dc4d9d9ffe3a875a799fc7 P b84b90efbbdb40bebe1b5adb43e9aa69 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a85606f73667466f804f9495aab892b4 in term 1.
I20260809 22:53:23.591895 28804 raft_consensus.cc:2454] T 33861f2c19dc4d9d9ffe3a875a799fc7 P ef758642c9de4d939a89e3d634d6a636 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a85606f73667466f804f9495aab892b4 in term 1.
I20260809 22:53:23.592237 28469 leader_election.cc:302] T 33861f2c19dc4d9d9ffe3a875a799fc7 P a85606f73667466f804f9495aab892b4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a85606f73667466f804f9495aab892b4, b84b90efbbdb40bebe1b5adb43e9aa69; no voters:
I20260809 22:53:23.592514 28909 raft_consensus.cc:2781] T 33861f2c19dc4d9d9ffe3a875a799fc7 P a85606f73667466f804f9495aab892b4 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 22:53:23.592855 28909 raft_consensus.cc:686] T 33861f2c19dc4d9d9ffe3a875a799fc7 P a85606f73667466f804f9495aab892b4 [term 1 LEADER]: Becoming Leader. State: Replica: a85606f73667466f804f9495aab892b4, State: Running, Role: LEADER
I20260809 22:53:23.593272 28909 consensus_queue.cc:237] T 33861f2c19dc4d9d9ffe3a875a799fc7 P a85606f73667466f804f9495aab892b4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b84b90efbbdb40bebe1b5adb43e9aa69" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 42763 } } peers { permanent_uuid: "ef758642c9de4d939a89e3d634d6a636" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 41323 } } peers { permanent_uuid: "a85606f73667466f804f9495aab892b4" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 46333 } }
I20260809 22:53:23.595162 28967 tablet_replica.cc:439] T 33861f2c19dc4d9d9ffe3a875a799fc7 P a85606f73667466f804f9495aab892b4: TxnStatusTablet state changed. Reason: New leader a85606f73667466f804f9495aab892b4. Latest consensus state: current_term: 1 leader_uuid: "a85606f73667466f804f9495aab892b4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b84b90efbbdb40bebe1b5adb43e9aa69" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 42763 } } peers { permanent_uuid: "ef758642c9de4d939a89e3d634d6a636" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 41323 } } peers { permanent_uuid: "a85606f73667466f804f9495aab892b4" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 46333 } } }
I20260809 22:53:23.595575 28967 tablet_replica.cc:442] T 33861f2c19dc4d9d9ffe3a875a799fc7 P a85606f73667466f804f9495aab892b4: This TxnStatusTablet replica's current role is: LEADER
I20260809 22:53:23.596264 28969 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260809 22:53:23.596505 28969 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260809 22:53:23.596877 28127 catalog_manager.cc:5495] T 33861f2c19dc4d9d9ffe3a875a799fc7 P a85606f73667466f804f9495aab892b4 reported cstate change: term changed from 0 to 1, leader changed from <none> to a85606f73667466f804f9495aab892b4 (127.15.113.130). New cstate: current_term: 1 leader_uuid: "a85606f73667466f804f9495aab892b4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b84b90efbbdb40bebe1b5adb43e9aa69" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 42763 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ef758642c9de4d939a89e3d634d6a636" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 41323 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a85606f73667466f804f9495aab892b4" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 46333 } health_report { overall_health: HEALTHY } } }
I20260809 22:53:23.623903 28974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:23.616769 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37308 (local address 127.15.113.130:46333)
0809 22:53:23.617610 (+ 841us) server_negotiation.cc:206] Beginning negotiation
0809 22:53:23.617613 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 22:53:23.617632 (+ 19us) server_negotiation.cc:407] Connection header received
0809 22:53:23.617707 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:53:23.617710 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:53:23.617802 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:53:23.617933 (+ 131us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 22:53:23.618411 (+ 478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:23.619299 (+ 888us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:23.621263 (+ 1964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:23.621779 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:23.621872 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:23.622581 (+ 709us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 22:53:23.623272 (+ 691us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 22:53:23.623383 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0809 22:53:23.623541 (+ 158us) server_negotiation.cc:299] Negotiation successful
0809 22:53:23.623617 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":727,"thread_start_us":569,"threads_started":1}
I20260809 22:53:23.623991 28953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:23.616767 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.130:46333 (local address 127.0.0.1:37308)
0809 22:53:23.617003 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0809 22:53:23.617024 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0809 22:53:23.617140 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:53:23.617998 (+ 858us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:53:23.618004 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:53:23.618020 (+ 16us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 22:53:23.618238 (+ 218us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:53:23.618251 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:53:23.619437 (+ 1186us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:53:23.619443 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:53:23.621102 (+ 1659us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:53:23.621136 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:53:23.621958 (+ 822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:53:23.621965 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:53:23.622221 (+ 256us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:23.622302 (+ 81us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 22:53:23.623396 (+ 1094us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 22:53:23.623420 (+ 24us) client_negotiation.cc:769] Sending connection context
0809 22:53:23.623513 (+ 93us) client_negotiation.cc:240] Negotiation successful
0809 22:53:23.623673 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":47}
I20260809 22:53:23.628576 28366 raft_consensus.cc:1261] T 33861f2c19dc4d9d9ffe3a875a799fc7 P b84b90efbbdb40bebe1b5adb43e9aa69 [term 1 FOLLOWER]: Refusing update from remote peer a85606f73667466f804f9495aab892b4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 22:53:23.628778 28804 raft_consensus.cc:1261] T 33861f2c19dc4d9d9ffe3a875a799fc7 P ef758642c9de4d939a89e3d634d6a636 [term 1 FOLLOWER]: Refusing update from remote peer a85606f73667466f804f9495aab892b4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 22:53:23.629452 28967 consensus_queue.cc:1035] T 33861f2c19dc4d9d9ffe3a875a799fc7 P a85606f73667466f804f9495aab892b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b84b90efbbdb40bebe1b5adb43e9aa69" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 42763 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 22:53:23.629838 28909 consensus_queue.cc:1035] T 33861f2c19dc4d9d9ffe3a875a799fc7 P a85606f73667466f804f9495aab892b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef758642c9de4d939a89e3d634d6a636" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 41323 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 22:53:23.633404 28911 tablet_replica.cc:439] T 33861f2c19dc4d9d9ffe3a875a799fc7 P b84b90efbbdb40bebe1b5adb43e9aa69: TxnStatusTablet state changed. Reason: New leader a85606f73667466f804f9495aab892b4. Latest consensus state: current_term: 1 leader_uuid: "a85606f73667466f804f9495aab892b4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b84b90efbbdb40bebe1b5adb43e9aa69" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 42763 } } peers { permanent_uuid: "ef758642c9de4d939a89e3d634d6a636" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 41323 } } peers { permanent_uuid: "a85606f73667466f804f9495aab892b4" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 46333 } } }
I20260809 22:53:23.633711 28911 tablet_replica.cc:442] T 33861f2c19dc4d9d9ffe3a875a799fc7 P b84b90efbbdb40bebe1b5adb43e9aa69: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 22:53:23.636132 28967 tablet_replica.cc:439] T 33861f2c19dc4d9d9ffe3a875a799fc7 P a85606f73667466f804f9495aab892b4: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "a85606f73667466f804f9495aab892b4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b84b90efbbdb40bebe1b5adb43e9aa69" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 42763 } } peers { permanent_uuid: "ef758642c9de4d939a89e3d634d6a636" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 41323 } } peers { permanent_uuid: "a85606f73667466f804f9495aab892b4" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 46333 } } }
I20260809 22:53:23.636397 28967 tablet_replica.cc:442] T 33861f2c19dc4d9d9ffe3a875a799fc7 P a85606f73667466f804f9495aab892b4: This TxnStatusTablet replica's current role is: LEADER
I20260809 22:53:23.639460 28976 mvcc.cc:205] Tried to move back new op lower bound from 7316750350854696960 to 7316750350721355776. Current Snapshot: MvccSnapshot[applied={T|T < 7316750350854696960}]
I20260809 22:53:23.649567 28914 tablet_replica.cc:439] T 33861f2c19dc4d9d9ffe3a875a799fc7 P ef758642c9de4d939a89e3d634d6a636: TxnStatusTablet state changed. Reason: New leader a85606f73667466f804f9495aab892b4. Latest consensus state: current_term: 1 leader_uuid: "a85606f73667466f804f9495aab892b4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b84b90efbbdb40bebe1b5adb43e9aa69" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 42763 } } peers { permanent_uuid: "ef758642c9de4d939a89e3d634d6a636" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 41323 } } peers { permanent_uuid: "a85606f73667466f804f9495aab892b4" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 46333 } } }
I20260809 22:53:23.649875 28914 tablet_replica.cc:442] T 33861f2c19dc4d9d9ffe3a875a799fc7 P ef758642c9de4d939a89e3d634d6a636: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 22:53:23.654171 28909 tablet_replica.cc:439] T 33861f2c19dc4d9d9ffe3a875a799fc7 P a85606f73667466f804f9495aab892b4: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "a85606f73667466f804f9495aab892b4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b84b90efbbdb40bebe1b5adb43e9aa69" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 42763 } } peers { permanent_uuid: "ef758642c9de4d939a89e3d634d6a636" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 41323 } } peers { permanent_uuid: "a85606f73667466f804f9495aab892b4" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 46333 } } }
I20260809 22:53:23.654518 28909 tablet_replica.cc:442] T 33861f2c19dc4d9d9ffe3a875a799fc7 P a85606f73667466f804f9495aab892b4: This TxnStatusTablet replica's current role is: LEADER
I20260809 22:53:23.658491 28911 tablet_replica.cc:439] T 33861f2c19dc4d9d9ffe3a875a799fc7 P b84b90efbbdb40bebe1b5adb43e9aa69: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "a85606f73667466f804f9495aab892b4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b84b90efbbdb40bebe1b5adb43e9aa69" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 42763 } } peers { permanent_uuid: "ef758642c9de4d939a89e3d634d6a636" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 41323 } } peers { permanent_uuid: "a85606f73667466f804f9495aab892b4" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 46333 } } }
I20260809 22:53:23.658785 28911 tablet_replica.cc:442] T 33861f2c19dc4d9d9ffe3a875a799fc7 P b84b90efbbdb40bebe1b5adb43e9aa69: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 22:53:23.666394 28914 tablet_replica.cc:439] T 33861f2c19dc4d9d9ffe3a875a799fc7 P ef758642c9de4d939a89e3d634d6a636: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "a85606f73667466f804f9495aab892b4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b84b90efbbdb40bebe1b5adb43e9aa69" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 42763 } } peers { permanent_uuid: "ef758642c9de4d939a89e3d634d6a636" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 41323 } } peers { permanent_uuid: "a85606f73667466f804f9495aab892b4" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 46333 } } }
I20260809 22:53:23.666662 28914 tablet_replica.cc:442] T 33861f2c19dc4d9d9ffe3a875a799fc7 P ef758642c9de4d939a89e3d634d6a636: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 22:53:23.677016 28991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:23.670349 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.130:46333 (local address 127.15.113.131:59627)
0809 22:53:23.671431 (+ 1082us) negotiation.cc:107] Waiting for socket to connect
0809 22:53:23.671452 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0809 22:53:23.671567 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:53:23.671873 (+ 306us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:53:23.671875 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:53:23.671885 (+ 10us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 22:53:23.672149 (+ 264us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:53:23.672158 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:53:23.673687 (+ 1529us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:53:23.673691 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:53:23.674724 (+ 1033us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:53:23.674735 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:53:23.675818 (+ 1083us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:53:23.675824 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:53:23.675957 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:23.676014 (+ 57us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 22:53:23.676475 (+ 461us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 22:53:23.676485 (+ 10us) client_negotiation.cc:769] Sending connection context
0809 22:53:23.676560 (+ 75us) client_negotiation.cc:240] Negotiation successful
0809 22:53:23.676665 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":921,"thread_start_us":801,"threads_started":1}
I20260809 22:53:23.677037 28974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:23.670400 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.113.131:59627 (local address 127.15.113.130:46333)
0809 22:53:23.670574 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0809 22:53:23.670580 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:53:23.671536 (+ 956us) server_negotiation.cc:407] Connection header received
0809 22:53:23.671666 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:53:23.671672 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:53:23.671760 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:53:23.671861 (+ 101us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 22:53:23.672284 (+ 423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:23.673549 (+ 1265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:23.674864 (+ 1315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:23.675610 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:23.675759 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:23.676198 (+ 439us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 22:53:23.676349 (+ 151us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 22:53:23.676468 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0809 22:53:23.676596 (+ 128us) server_negotiation.cc:299] Negotiation successful
0809 22:53:23.676686 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260809 22:53:23.686748 28597 raft_consensus.cc:1261] T 9c6278eaa9d5483e9757cc1755a11122 P a85606f73667466f804f9495aab892b4 [term 1 FOLLOWER]: Refusing update from remote peer ef758642c9de4d939a89e3d634d6a636: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 22:53:23.686740 28366 raft_consensus.cc:1261] T 9c6278eaa9d5483e9757cc1755a11122 P b84b90efbbdb40bebe1b5adb43e9aa69 [term 1 FOLLOWER]: Refusing update from remote peer ef758642c9de4d939a89e3d634d6a636: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 22:53:23.687377 28932 consensus_queue.cc:1035] T 9c6278eaa9d5483e9757cc1755a11122 P ef758642c9de4d939a89e3d634d6a636 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b84b90efbbdb40bebe1b5adb43e9aa69" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 42763 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 22:53:23.687822 28914 consensus_queue.cc:1035] T 9c6278eaa9d5483e9757cc1755a11122 P ef758642c9de4d939a89e3d634d6a636 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a85606f73667466f804f9495aab892b4" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 46333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 22:53:23.701761 28978 mvcc.cc:205] Tried to move back new op lower bound from 7316750351095767040 to 7316750349521649664. Current Snapshot: MvccSnapshot[applied={T|T < 7316750351095767040}]
I20260809 22:53:23.734334 28965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:23.726986 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.113.130:35181 (local address 127.15.113.131:41323)
0809 22:53:23.727192 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0809 22:53:23.727199 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:53:23.727972 (+ 773us) server_negotiation.cc:407] Connection header received
0809 22:53:23.728292 (+ 320us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:53:23.728298 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:53:23.728389 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:53:23.728495 (+ 106us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 22:53:23.729439 (+ 944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:23.730226 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:23.732461 (+ 2235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:23.732924 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:23.733057 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:23.733605 (+ 548us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 22:53:23.733714 (+ 109us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 22:53:23.733793 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0809 22:53:23.733990 (+ 197us) server_negotiation.cc:299] Negotiation successful
0809 22:53:23.734067 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260809 22:53:23.734544 28999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:23.727075 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.131:41323 (local address 127.15.113.130:35181)
0809 22:53:23.727859 (+ 784us) negotiation.cc:107] Waiting for socket to connect
0809 22:53:23.727881 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0809 22:53:23.728101 (+ 220us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:53:23.728768 (+ 667us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:53:23.728773 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:53:23.728791 (+ 18us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 22:53:23.729277 (+ 486us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:53:23.729291 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:53:23.730364 (+ 1073us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:53:23.730370 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:53:23.732258 (+ 1888us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:53:23.732277 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:53:23.733121 (+ 844us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:53:23.733129 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:53:23.733342 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:23.733421 (+ 79us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 22:53:23.733859 (+ 438us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 22:53:23.733884 (+ 25us) client_negotiation.cc:769] Sending connection context
0809 22:53:23.733955 (+ 71us) client_negotiation.cc:240] Negotiation successful
0809 22:53:23.734092 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":544,"thread_start_us":366,"threads_started":1}
I20260809 22:53:23.767298 28128 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:49134:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 22:53:23.767596 28128 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:49134:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 22:53:23.770771 28128 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f37bab48ada649e6bd170b082f14874f: Sending DeleteTablet for 3 replicas of tablet 9c6278eaa9d5483e9757cc1755a11122
I20260809 22:53:23.771467 28330 tablet_service.cc:1514] Processing DeleteTablet for tablet 9c6278eaa9d5483e9757cc1755a11122 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:53:23 UTC) from {username='oryx', principal='oryx/127.15.113.190@KRBTEST.COM'} at 127.0.0.1:35596
I20260809 22:53:23.771517 28549 tablet_service.cc:1514] Processing DeleteTablet for tablet 9c6278eaa9d5483e9757cc1755a11122 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:53:23 UTC) from {username='oryx', principal='oryx/127.15.113.190@KRBTEST.COM'} at 127.0.0.1:37298
I20260809 22:53:23.772384 28765 tablet_service.cc:1514] Processing DeleteTablet for tablet 9c6278eaa9d5483e9757cc1755a11122 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:53:23 UTC) from {username='oryx', principal='oryx/127.15.113.190@KRBTEST.COM'} at 127.0.0.1:49762
I20260809 22:53:23.780303 29009 tablet_replica.cc:330] T 9c6278eaa9d5483e9757cc1755a11122 P ef758642c9de4d939a89e3d634d6a636: stopping tablet replica
I20260809 22:53:23.780498 29009 raft_consensus.cc:2229] T 9c6278eaa9d5483e9757cc1755a11122 P ef758642c9de4d939a89e3d634d6a636 [term 1 LEADER]: Raft consensus shutting down.
I20260809 22:53:23.780912 29009 raft_consensus.cc:2258] T 9c6278eaa9d5483e9757cc1755a11122 P ef758642c9de4d939a89e3d634d6a636 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 22:53:23.782359 29007 tablet_replica.cc:330] T 9c6278eaa9d5483e9757cc1755a11122 P a85606f73667466f804f9495aab892b4: stopping tablet replica
I20260809 22:53:23.783535 29009 ts_tablet_manager.cc:1905] T 9c6278eaa9d5483e9757cc1755a11122 P ef758642c9de4d939a89e3d634d6a636: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 22:53:23.783811 29007 raft_consensus.cc:2229] T 9c6278eaa9d5483e9757cc1755a11122 P a85606f73667466f804f9495aab892b4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 22:53:23.784163 29007 raft_consensus.cc:2258] T 9c6278eaa9d5483e9757cc1755a11122 P a85606f73667466f804f9495aab892b4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 22:53:23.791304 29009 ts_tablet_manager.cc:1918] T 9c6278eaa9d5483e9757cc1755a11122 P ef758642c9de4d939a89e3d634d6a636: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 22:53:23.792325 29007 ts_tablet_manager.cc:1905] T 9c6278eaa9d5483e9757cc1755a11122 P a85606f73667466f804f9495aab892b4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 22:53:23.792797 29006 tablet_replica.cc:330] T 9c6278eaa9d5483e9757cc1755a11122 P b84b90efbbdb40bebe1b5adb43e9aa69: stopping tablet replica
I20260809 22:53:23.797111 29006 raft_consensus.cc:2229] T 9c6278eaa9d5483e9757cc1755a11122 P b84b90efbbdb40bebe1b5adb43e9aa69 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 22:53:23.797386 29006 raft_consensus.cc:2258] T 9c6278eaa9d5483e9757cc1755a11122 P b84b90efbbdb40bebe1b5adb43e9aa69 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 22:53:23.797652 29009 log.cc:1196] T 9c6278eaa9d5483e9757cc1755a11122 P ef758642c9de4d939a89e3d634d6a636: Deleting WAL directory at /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315935617919-15814-0/minicluster-data/ts-2/wal/wals/9c6278eaa9d5483e9757cc1755a11122
I20260809 22:53:23.798171 29009 ts_tablet_manager.cc:1939] T 9c6278eaa9d5483e9757cc1755a11122 P ef758642c9de4d939a89e3d634d6a636: Deleting consensus metadata
I20260809 22:53:23.798506 29006 ts_tablet_manager.cc:1905] T 9c6278eaa9d5483e9757cc1755a11122 P b84b90efbbdb40bebe1b5adb43e9aa69: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 22:53:23.799247 28106 catalog_manager.cc:4841] TS ef758642c9de4d939a89e3d634d6a636 (127.15.113.131:41323): tablet 9c6278eaa9d5483e9757cc1755a11122 (table test-table [id=6cfc3601a2b44b029db05ee339f2e62b]) successfully deleted
I20260809 22:53:23.800268 29007 ts_tablet_manager.cc:1918] T 9c6278eaa9d5483e9757cc1755a11122 P a85606f73667466f804f9495aab892b4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 22:53:23.800406 29007 log.cc:1196] T 9c6278eaa9d5483e9757cc1755a11122 P a85606f73667466f804f9495aab892b4: Deleting WAL directory at /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315935617919-15814-0/minicluster-data/ts-1/wal/wals/9c6278eaa9d5483e9757cc1755a11122
I20260809 22:53:23.800956 29007 ts_tablet_manager.cc:1939] T 9c6278eaa9d5483e9757cc1755a11122 P a85606f73667466f804f9495aab892b4: Deleting consensus metadata
I20260809 22:53:23.802028 28103 catalog_manager.cc:4841] TS a85606f73667466f804f9495aab892b4 (127.15.113.130:46333): tablet 9c6278eaa9d5483e9757cc1755a11122 (table test-table [id=6cfc3601a2b44b029db05ee339f2e62b]) successfully deleted
I20260809 22:53:23.803709 29006 ts_tablet_manager.cc:1918] T 9c6278eaa9d5483e9757cc1755a11122 P b84b90efbbdb40bebe1b5adb43e9aa69: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 22:53:23.803825 29006 log.cc:1196] T 9c6278eaa9d5483e9757cc1755a11122 P b84b90efbbdb40bebe1b5adb43e9aa69: Deleting WAL directory at /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315935617919-15814-0/minicluster-data/ts-0/wal/wals/9c6278eaa9d5483e9757cc1755a11122
I20260809 22:53:23.804271 29006 ts_tablet_manager.cc:1939] T 9c6278eaa9d5483e9757cc1755a11122 P b84b90efbbdb40bebe1b5adb43e9aa69: Deleting consensus metadata
I20260809 22:53:23.805231 28102 catalog_manager.cc:4841] TS b84b90efbbdb40bebe1b5adb43e9aa69 (127.15.113.129:42763): tablet 9c6278eaa9d5483e9757cc1755a11122 (table test-table [id=6cfc3601a2b44b029db05ee339f2e62b]) successfully deleted
Aug 09 22:53:23 dist-test-slave-ncp9 krb5kdc[28066](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316003, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 09 22:53:23 dist-test-slave-ncp9 krb5kdc[28066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.15.113.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 22:53:23 dist-test-slave-ncp9 krb5kdc[28066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.15.113.190@KRBTEST.COM, Server not found in Kerberos database
W20260809 22:53:23.850556 28954 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:53:23.839004 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49156 (local address 127.15.113.190:34881)
0809 22:53:23.839240 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0809 22:53:23.839247 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:53:23.839567 (+ 320us) server_negotiation.cc:407] Connection header received
0809 22:53:23.839752 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:53:23.839756 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:53:23.839837 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:53:23.839922 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:53:23.841574 (+ 1652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:23.842322 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:23.844189 (+ 1867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:23.844827 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:23.844983 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:23.850301 (+ 5318us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49156: BlockingRecv error: recv got EOF from 127.0.0.1:49156 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
Aug 09 22:53:23 dist-test-slave-ncp9 krb5kdc[28066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316003, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.15.113.190@KRBTEST.COM
I20260809 22:53:23.868448 28954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:23.853446 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49160 (local address 127.15.113.190:34881)
0809 22:53:23.853635 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0809 22:53:23.853643 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 22:53:23.853985 (+ 342us) server_negotiation.cc:407] Connection header received
0809 22:53:23.854149 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:53:23.854152 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:53:23.854212 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:53:23.854285 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:53:23.855799 (+ 1514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:23.856504 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:23.858364 (+ 1860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:23.858795 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:23.858888 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:23.863418 (+ 4530us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:53:23.863423 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:53:23.863426 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:53:23.863482 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:53:23.866130 (+ 2648us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:53:23.866837 (+ 707us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:53:23.866841 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:53:23.866843 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:53:23.866954 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:53:23.867283 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:53:23.867285 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:53:23.867287 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:53:23.867546 (+ 259us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:53:23.867745 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0809 22:53:23.868016 (+ 271us) server_negotiation.cc:299] Negotiation successful
0809 22:53:23.868181 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260809 22:53:23.871847 28128 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49160:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 22:53:23.872581 28128 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 22:53:23.879637 28549 tablet_service.cc:1467] Processing CreateTablet for tablet 65390bf0180040dc933ca17082a30b20 (DEFAULT_TABLE table=test-table [id=a56b5d39a2b94e418eca77879c331bab]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:53:23.879637 28330 tablet_service.cc:1467] Processing CreateTablet for tablet 65390bf0180040dc933ca17082a30b20 (DEFAULT_TABLE table=test-table [id=a56b5d39a2b94e418eca77879c331bab]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:53:23.879838 28765 tablet_service.cc:1467] Processing CreateTablet for tablet 65390bf0180040dc933ca17082a30b20 (DEFAULT_TABLE table=test-table [id=a56b5d39a2b94e418eca77879c331bab]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:53:23.880071 28330 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 65390bf0180040dc933ca17082a30b20. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:53:23.880080 28549 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 65390bf0180040dc933ca17082a30b20. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:53:23.880259 28765 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 65390bf0180040dc933ca17082a30b20. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:53:23.892455 29031 tablet_bootstrap.cc:492] T 65390bf0180040dc933ca17082a30b20 P a85606f73667466f804f9495aab892b4: Bootstrap starting.
I20260809 22:53:23.892812 29028 tablet_bootstrap.cc:492] T 65390bf0180040dc933ca17082a30b20 P b84b90efbbdb40bebe1b5adb43e9aa69: Bootstrap starting.
I20260809 22:53:23.894465 29031 tablet_bootstrap.cc:654] T 65390bf0180040dc933ca17082a30b20 P a85606f73667466f804f9495aab892b4: Neither blocks nor log segments found. Creating new log.
I20260809 22:53:23.894924 29028 tablet_bootstrap.cc:654] T 65390bf0180040dc933ca17082a30b20 P b84b90efbbdb40bebe1b5adb43e9aa69: Neither blocks nor log segments found. Creating new log.
I20260809 22:53:23.896265 29029 tablet_bootstrap.cc:492] T 65390bf0180040dc933ca17082a30b20 P ef758642c9de4d939a89e3d634d6a636: Bootstrap starting.
I20260809 22:53:23.897061 29028 tablet_bootstrap.cc:492] T 65390bf0180040dc933ca17082a30b20 P b84b90efbbdb40bebe1b5adb43e9aa69: No bootstrap required, opened a new log
I20260809 22:53:23.897266 29028 ts_tablet_manager.cc:1397] T 65390bf0180040dc933ca17082a30b20 P b84b90efbbdb40bebe1b5adb43e9aa69: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260809 22:53:23.898012 29031 tablet_bootstrap.cc:492] T 65390bf0180040dc933ca17082a30b20 P a85606f73667466f804f9495aab892b4: No bootstrap required, opened a new log
I20260809 22:53:23.897899 29028 raft_consensus.cc:348] T 65390bf0180040dc933ca17082a30b20 P b84b90efbbdb40bebe1b5adb43e9aa69 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b84b90efbbdb40bebe1b5adb43e9aa69" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 42763 } } peers { permanent_uuid: "a85606f73667466f804f9495aab892b4" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 46333 } } peers { permanent_uuid: "ef758642c9de4d939a89e3d634d6a636" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 41323 } }
I20260809 22:53:23.898098 29028 raft_consensus.cc:374] T 65390bf0180040dc933ca17082a30b20 P b84b90efbbdb40bebe1b5adb43e9aa69 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:53:23.898160 29031 ts_tablet_manager.cc:1397] T 65390bf0180040dc933ca17082a30b20 P a85606f73667466f804f9495aab892b4: Time spent bootstrapping tablet: real 0.006s user 0.003s sys 0.000s
I20260809 22:53:23.898196 29028 raft_consensus.cc:729] T 65390bf0180040dc933ca17082a30b20 P b84b90efbbdb40bebe1b5adb43e9aa69 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b84b90efbbdb40bebe1b5adb43e9aa69, State: Initialized, Role: FOLLOWER
I20260809 22:53:23.898413 29028 consensus_queue.cc:260] T 65390bf0180040dc933ca17082a30b20 P b84b90efbbdb40bebe1b5adb43e9aa69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b84b90efbbdb40bebe1b5adb43e9aa69" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 42763 } } peers { permanent_uuid: "a85606f73667466f804f9495aab892b4" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 46333 } } peers { permanent_uuid: "ef758642c9de4d939a89e3d634d6a636" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 41323 } }
I20260809 22:53:23.898983 29028 ts_tablet_manager.cc:1428] T 65390bf0180040dc933ca17082a30b20 P b84b90efbbdb40bebe1b5adb43e9aa69: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260809 22:53:23.898895 29031 raft_consensus.cc:348] T 65390bf0180040dc933ca17082a30b20 P a85606f73667466f804f9495aab892b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b84b90efbbdb40bebe1b5adb43e9aa69" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 42763 } } peers { permanent_uuid: "a85606f73667466f804f9495aab892b4" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 46333 } } peers { permanent_uuid: "ef758642c9de4d939a89e3d634d6a636" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 41323 } }
I20260809 22:53:23.899163 29031 raft_consensus.cc:374] T 65390bf0180040dc933ca17082a30b20 P a85606f73667466f804f9495aab892b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:53:23.899255 29031 raft_consensus.cc:729] T 65390bf0180040dc933ca17082a30b20 P a85606f73667466f804f9495aab892b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a85606f73667466f804f9495aab892b4, State: Initialized, Role: FOLLOWER
I20260809 22:53:23.899502 29031 consensus_queue.cc:260] T 65390bf0180040dc933ca17082a30b20 P a85606f73667466f804f9495aab892b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b84b90efbbdb40bebe1b5adb43e9aa69" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 42763 } } peers { permanent_uuid: "a85606f73667466f804f9495aab892b4" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 46333 } } peers { permanent_uuid: "ef758642c9de4d939a89e3d634d6a636" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 41323 } }
I20260809 22:53:23.900776 29029 tablet_bootstrap.cc:654] T 65390bf0180040dc933ca17082a30b20 P ef758642c9de4d939a89e3d634d6a636: Neither blocks nor log segments found. Creating new log.
I20260809 22:53:23.902047 29031 ts_tablet_manager.cc:1428] T 65390bf0180040dc933ca17082a30b20 P a85606f73667466f804f9495aab892b4: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260809 22:53:23.904693 29029 tablet_bootstrap.cc:492] T 65390bf0180040dc933ca17082a30b20 P ef758642c9de4d939a89e3d634d6a636: No bootstrap required, opened a new log
I20260809 22:53:23.904805 29029 ts_tablet_manager.cc:1397] T 65390bf0180040dc933ca17082a30b20 P ef758642c9de4d939a89e3d634d6a636: Time spent bootstrapping tablet: real 0.009s user 0.004s sys 0.000s
I20260809 22:53:23.905375 29029 raft_consensus.cc:348] T 65390bf0180040dc933ca17082a30b20 P ef758642c9de4d939a89e3d634d6a636 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b84b90efbbdb40bebe1b5adb43e9aa69" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 42763 } } peers { permanent_uuid: "a85606f73667466f804f9495aab892b4" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 46333 } } peers { permanent_uuid: "ef758642c9de4d939a89e3d634d6a636" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 41323 } }
I20260809 22:53:23.905556 29029 raft_consensus.cc:374] T 65390bf0180040dc933ca17082a30b20 P ef758642c9de4d939a89e3d634d6a636 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:53:23.905640 29029 raft_consensus.cc:729] T 65390bf0180040dc933ca17082a30b20 P ef758642c9de4d939a89e3d634d6a636 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef758642c9de4d939a89e3d634d6a636, State: Initialized, Role: FOLLOWER
I20260809 22:53:23.905830 29029 consensus_queue.cc:260] T 65390bf0180040dc933ca17082a30b20 P ef758642c9de4d939a89e3d634d6a636 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b84b90efbbdb40bebe1b5adb43e9aa69" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 42763 } } peers { permanent_uuid: "a85606f73667466f804f9495aab892b4" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 46333 } } peers { permanent_uuid: "ef758642c9de4d939a89e3d634d6a636" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 41323 } }
I20260809 22:53:23.907071 29029 ts_tablet_manager.cc:1428] T 65390bf0180040dc933ca17082a30b20 P ef758642c9de4d939a89e3d634d6a636: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260809 22:53:24.196504 28992 raft_consensus.cc:482] T 65390bf0180040dc933ca17082a30b20 P a85606f73667466f804f9495aab892b4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 22:53:24.196682 28992 raft_consensus.cc:504] T 65390bf0180040dc933ca17082a30b20 P a85606f73667466f804f9495aab892b4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b84b90efbbdb40bebe1b5adb43e9aa69" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 42763 } } peers { permanent_uuid: "a85606f73667466f804f9495aab892b4" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 46333 } } peers { permanent_uuid: "ef758642c9de4d939a89e3d634d6a636" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 41323 } }
I20260809 22:53:24.197281 28992 leader_election.cc:288] T 65390bf0180040dc933ca17082a30b20 P a85606f73667466f804f9495aab892b4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b84b90efbbdb40bebe1b5adb43e9aa69 (127.15.113.129:42763), ef758642c9de4d939a89e3d634d6a636 (127.15.113.131:41323)
I20260809 22:53:24.197672 28366 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "65390bf0180040dc933ca17082a30b20" candidate_uuid: "a85606f73667466f804f9495aab892b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b84b90efbbdb40bebe1b5adb43e9aa69" is_pre_election: true
I20260809 22:53:24.197789 28802 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "65390bf0180040dc933ca17082a30b20" candidate_uuid: "a85606f73667466f804f9495aab892b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef758642c9de4d939a89e3d634d6a636" is_pre_election: true
I20260809 22:53:24.197903 28366 raft_consensus.cc:2454] T 65390bf0180040dc933ca17082a30b20 P b84b90efbbdb40bebe1b5adb43e9aa69 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a85606f73667466f804f9495aab892b4 in term 0.
I20260809 22:53:24.198032 28802 raft_consensus.cc:2454] T 65390bf0180040dc933ca17082a30b20 P ef758642c9de4d939a89e3d634d6a636 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a85606f73667466f804f9495aab892b4 in term 0.
I20260809 22:53:24.198324 28469 leader_election.cc:302] T 65390bf0180040dc933ca17082a30b20 P a85606f73667466f804f9495aab892b4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a85606f73667466f804f9495aab892b4, b84b90efbbdb40bebe1b5adb43e9aa69; no voters:
I20260809 22:53:24.198559 28992 raft_consensus.cc:2781] T 65390bf0180040dc933ca17082a30b20 P a85606f73667466f804f9495aab892b4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 22:53:24.198676 28992 raft_consensus.cc:482] T 65390bf0180040dc933ca17082a30b20 P a85606f73667466f804f9495aab892b4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 22:53:24.198748 28992 raft_consensus.cc:3037] T 65390bf0180040dc933ca17082a30b20 P a85606f73667466f804f9495aab892b4 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:53:24.200814 28992 raft_consensus.cc:504] T 65390bf0180040dc933ca17082a30b20 P a85606f73667466f804f9495aab892b4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b84b90efbbdb40bebe1b5adb43e9aa69" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 42763 } } peers { permanent_uuid: "a85606f73667466f804f9495aab892b4" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 46333 } } peers { permanent_uuid: "ef758642c9de4d939a89e3d634d6a636" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 41323 } }
I20260809 22:53:24.201352 28992 leader_election.cc:288] T 65390bf0180040dc933ca17082a30b20 P a85606f73667466f804f9495aab892b4 [CANDIDATE]: Term 1 election: Requested vote from peers b84b90efbbdb40bebe1b5adb43e9aa69 (127.15.113.129:42763), ef758642c9de4d939a89e3d634d6a636 (127.15.113.131:41323)
I20260809 22:53:24.201774 28366 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "65390bf0180040dc933ca17082a30b20" candidate_uuid: "a85606f73667466f804f9495aab892b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b84b90efbbdb40bebe1b5adb43e9aa69"
I20260809 22:53:24.201841 28802 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "65390bf0180040dc933ca17082a30b20" candidate_uuid: "a85606f73667466f804f9495aab892b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef758642c9de4d939a89e3d634d6a636"
I20260809 22:53:24.201987 28366 raft_consensus.cc:3037] T 65390bf0180040dc933ca17082a30b20 P b84b90efbbdb40bebe1b5adb43e9aa69 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:53:24.202021 28802 raft_consensus.cc:3037] T 65390bf0180040dc933ca17082a30b20 P ef758642c9de4d939a89e3d634d6a636 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:53:24.204790 28802 raft_consensus.cc:2454] T 65390bf0180040dc933ca17082a30b20 P ef758642c9de4d939a89e3d634d6a636 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a85606f73667466f804f9495aab892b4 in term 1.
I20260809 22:53:24.204804 28366 raft_consensus.cc:2454] T 65390bf0180040dc933ca17082a30b20 P b84b90efbbdb40bebe1b5adb43e9aa69 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a85606f73667466f804f9495aab892b4 in term 1.
I20260809 22:53:24.205269 28469 leader_election.cc:302] T 65390bf0180040dc933ca17082a30b20 P a85606f73667466f804f9495aab892b4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: a85606f73667466f804f9495aab892b4, b84b90efbbdb40bebe1b5adb43e9aa69, ef758642c9de4d939a89e3d634d6a636; no voters:
I20260809 22:53:24.205554 28992 raft_consensus.cc:2781] T 65390bf0180040dc933ca17082a30b20 P a85606f73667466f804f9495aab892b4 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 22:53:24.205685 28992 raft_consensus.cc:686] T 65390bf0180040dc933ca17082a30b20 P a85606f73667466f804f9495aab892b4 [term 1 LEADER]: Becoming Leader. State: Replica: a85606f73667466f804f9495aab892b4, State: Running, Role: LEADER
I20260809 22:53:24.205921 28992 consensus_queue.cc:237] T 65390bf0180040dc933ca17082a30b20 P a85606f73667466f804f9495aab892b4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b84b90efbbdb40bebe1b5adb43e9aa69" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 42763 } } peers { permanent_uuid: "a85606f73667466f804f9495aab892b4" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 46333 } } peers { permanent_uuid: "ef758642c9de4d939a89e3d634d6a636" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 41323 } }
I20260809 22:53:24.208406 28128 catalog_manager.cc:5495] T 65390bf0180040dc933ca17082a30b20 P a85606f73667466f804f9495aab892b4 reported cstate change: term changed from 0 to 1, leader changed from <none> to a85606f73667466f804f9495aab892b4 (127.15.113.130). New cstate: current_term: 1 leader_uuid: "a85606f73667466f804f9495aab892b4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b84b90efbbdb40bebe1b5adb43e9aa69" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 42763 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a85606f73667466f804f9495aab892b4" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 46333 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ef758642c9de4d939a89e3d634d6a636" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 41323 } health_report { overall_health: UNKNOWN } } }
I20260809 22:53:24.260519 29040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:24.255794 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37314 (local address 127.15.113.130:46333)
0809 22:53:24.256201 (+ 407us) server_negotiation.cc:206] Beginning negotiation
0809 22:53:24.256206 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:53:24.256225 (+ 19us) server_negotiation.cc:407] Connection header received
0809 22:53:24.256288 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:53:24.256293 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:53:24.256364 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:53:24.256445 (+ 81us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 22:53:24.256823 (+ 378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:24.257579 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:24.258904 (+ 1325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:24.259287 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:24.259394 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:24.259837 (+ 443us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 22:53:24.259954 (+ 117us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 22:53:24.260027 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0809 22:53:24.260178 (+ 151us) server_negotiation.cc:299] Negotiation successful
0809 22:53:24.260273 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":301,"thread_start_us":203,"threads_started":1}
I20260809 22:53:24.263561 28366 raft_consensus.cc:1261] T 65390bf0180040dc933ca17082a30b20 P b84b90efbbdb40bebe1b5adb43e9aa69 [term 1 FOLLOWER]: Refusing update from remote peer a85606f73667466f804f9495aab892b4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 22:53:24.263559 28802 raft_consensus.cc:1261] T 65390bf0180040dc933ca17082a30b20 P ef758642c9de4d939a89e3d634d6a636 [term 1 FOLLOWER]: Refusing update from remote peer a85606f73667466f804f9495aab892b4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 22:53:24.264127 28994 consensus_queue.cc:1035] T 65390bf0180040dc933ca17082a30b20 P a85606f73667466f804f9495aab892b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef758642c9de4d939a89e3d634d6a636" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 41323 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 22:53:24.264423 28992 consensus_queue.cc:1035] T 65390bf0180040dc933ca17082a30b20 P a85606f73667466f804f9495aab892b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b84b90efbbdb40bebe1b5adb43e9aa69" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 42763 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 22:53:24.284710 28128 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49160:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 22:53:24.284972 28128 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49160:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 22:53:24.287665 28128 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f37bab48ada649e6bd170b082f14874f: Sending DeleteTablet for 3 replicas of tablet 65390bf0180040dc933ca17082a30b20
I20260809 22:53:24.288404 28330 tablet_service.cc:1514] Processing DeleteTablet for tablet 65390bf0180040dc933ca17082a30b20 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:53:24 UTC) from {username='oryx', principal='oryx/127.15.113.190@KRBTEST.COM'} at 127.0.0.1:35596
I20260809 22:53:24.288579 28765 tablet_service.cc:1514] Processing DeleteTablet for tablet 65390bf0180040dc933ca17082a30b20 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:53:24 UTC) from {username='oryx', principal='oryx/127.15.113.190@KRBTEST.COM'} at 127.0.0.1:49762
I20260809 22:53:24.288700 29006 tablet_replica.cc:330] T 65390bf0180040dc933ca17082a30b20 P b84b90efbbdb40bebe1b5adb43e9aa69: stopping tablet replica
I20260809 22:53:24.288781 28549 tablet_service.cc:1514] Processing DeleteTablet for tablet 65390bf0180040dc933ca17082a30b20 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:53:24 UTC) from {username='oryx', principal='oryx/127.15.113.190@KRBTEST.COM'} at 127.0.0.1:37298
I20260809 22:53:24.288932 29006 raft_consensus.cc:2229] T 65390bf0180040dc933ca17082a30b20 P b84b90efbbdb40bebe1b5adb43e9aa69 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 22:53:24.289048 29009 tablet_replica.cc:330] T 65390bf0180040dc933ca17082a30b20 P ef758642c9de4d939a89e3d634d6a636: stopping tablet replica
I20260809 22:53:24.289073 29007 tablet_replica.cc:330] T 65390bf0180040dc933ca17082a30b20 P a85606f73667466f804f9495aab892b4: stopping tablet replica
I20260809 22:53:24.289150 29006 raft_consensus.cc:2258] T 65390bf0180040dc933ca17082a30b20 P b84b90efbbdb40bebe1b5adb43e9aa69 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 22:53:24.289256 29009 raft_consensus.cc:2229] T 65390bf0180040dc933ca17082a30b20 P ef758642c9de4d939a89e3d634d6a636 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 22:53:24.289276 29007 raft_consensus.cc:2229] T 65390bf0180040dc933ca17082a30b20 P a85606f73667466f804f9495aab892b4 [term 1 LEADER]: Raft consensus shutting down.
I20260809 22:53:24.289453 29009 raft_consensus.cc:2258] T 65390bf0180040dc933ca17082a30b20 P ef758642c9de4d939a89e3d634d6a636 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 22:53:24.289757 29007 raft_consensus.cc:2258] T 65390bf0180040dc933ca17082a30b20 P a85606f73667466f804f9495aab892b4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 22:53:24.290486 29009 ts_tablet_manager.cc:1905] T 65390bf0180040dc933ca17082a30b20 P ef758642c9de4d939a89e3d634d6a636: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 22:53:24.290484 29006 ts_tablet_manager.cc:1905] T 65390bf0180040dc933ca17082a30b20 P b84b90efbbdb40bebe1b5adb43e9aa69: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 22:53:24.291944 29007 ts_tablet_manager.cc:1905] T 65390bf0180040dc933ca17082a30b20 P a85606f73667466f804f9495aab892b4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 22:53:24.296706 29006 ts_tablet_manager.cc:1918] T 65390bf0180040dc933ca17082a30b20 P b84b90efbbdb40bebe1b5adb43e9aa69: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 22:53:24.296705 29009 ts_tablet_manager.cc:1918] T 65390bf0180040dc933ca17082a30b20 P ef758642c9de4d939a89e3d634d6a636: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 22:53:24.296841 29009 log.cc:1196] T 65390bf0180040dc933ca17082a30b20 P ef758642c9de4d939a89e3d634d6a636: Deleting WAL directory at /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315935617919-15814-0/minicluster-data/ts-2/wal/wals/65390bf0180040dc933ca17082a30b20
I20260809 22:53:24.296839 29006 log.cc:1196] T 65390bf0180040dc933ca17082a30b20 P b84b90efbbdb40bebe1b5adb43e9aa69: Deleting WAL directory at /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315935617919-15814-0/minicluster-data/ts-0/wal/wals/65390bf0180040dc933ca17082a30b20
I20260809 22:53:24.297252 29009 ts_tablet_manager.cc:1939] T 65390bf0180040dc933ca17082a30b20 P ef758642c9de4d939a89e3d634d6a636: Deleting consensus metadata
I20260809 22:53:24.297266 29006 ts_tablet_manager.cc:1939] T 65390bf0180040dc933ca17082a30b20 P b84b90efbbdb40bebe1b5adb43e9aa69: Deleting consensus metadata
I20260809 22:53:24.297338 28128 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49160:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 22:53:24.297947 28128 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 22:53:24.298214 28102 catalog_manager.cc:4841] TS b84b90efbbdb40bebe1b5adb43e9aa69 (127.15.113.129:42763): tablet 65390bf0180040dc933ca17082a30b20 (table test-table [id=a56b5d39a2b94e418eca77879c331bab]) successfully deleted
I20260809 22:53:24.298431 28106 catalog_manager.cc:4841] TS ef758642c9de4d939a89e3d634d6a636 (127.15.113.131:41323): tablet 65390bf0180040dc933ca17082a30b20 (table test-table [id=a56b5d39a2b94e418eca77879c331bab]) successfully deleted
I20260809 22:53:24.301767 29007 ts_tablet_manager.cc:1918] T 65390bf0180040dc933ca17082a30b20 P a85606f73667466f804f9495aab892b4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 22:53:24.302239 29007 log.cc:1196] T 65390bf0180040dc933ca17082a30b20 P a85606f73667466f804f9495aab892b4: Deleting WAL directory at /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315935617919-15814-0/minicluster-data/ts-1/wal/wals/65390bf0180040dc933ca17082a30b20
I20260809 22:53:24.302762 29007 ts_tablet_manager.cc:1939] T 65390bf0180040dc933ca17082a30b20 P a85606f73667466f804f9495aab892b4: Deleting consensus metadata
I20260809 22:53:24.303908 28330 tablet_service.cc:1467] Processing CreateTablet for tablet 277c5f2bf11b4167af28ab8497bdcf9f (DEFAULT_TABLE table=test-table [id=30de39925fb448b48ee0ef0c28c37779]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:53:24.304270 28330 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 277c5f2bf11b4167af28ab8497bdcf9f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:53:24.304322 28549 tablet_service.cc:1467] Processing CreateTablet for tablet 277c5f2bf11b4167af28ab8497bdcf9f (DEFAULT_TABLE table=test-table [id=30de39925fb448b48ee0ef0c28c37779]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:53:24.304683 28549 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 277c5f2bf11b4167af28ab8497bdcf9f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:53:24.305436 28103 catalog_manager.cc:4841] TS a85606f73667466f804f9495aab892b4 (127.15.113.130:46333): tablet 65390bf0180040dc933ca17082a30b20 (table test-table [id=a56b5d39a2b94e418eca77879c331bab]) successfully deleted
I20260809 22:53:24.309584 28765 tablet_service.cc:1467] Processing CreateTablet for tablet 277c5f2bf11b4167af28ab8497bdcf9f (DEFAULT_TABLE table=test-table [id=30de39925fb448b48ee0ef0c28c37779]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:53:24.310003 28765 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 277c5f2bf11b4167af28ab8497bdcf9f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:53:24.310109 29028 tablet_bootstrap.cc:492] T 277c5f2bf11b4167af28ab8497bdcf9f P b84b90efbbdb40bebe1b5adb43e9aa69: Bootstrap starting.
I20260809 22:53:24.311471 29028 tablet_bootstrap.cc:654] T 277c5f2bf11b4167af28ab8497bdcf9f P b84b90efbbdb40bebe1b5adb43e9aa69: Neither blocks nor log segments found. Creating new log.
I20260809 22:53:24.313102 29031 tablet_bootstrap.cc:492] T 277c5f2bf11b4167af28ab8497bdcf9f P a85606f73667466f804f9495aab892b4: Bootstrap starting.
I20260809 22:53:24.313161 29028 tablet_bootstrap.cc:492] T 277c5f2bf11b4167af28ab8497bdcf9f P b84b90efbbdb40bebe1b5adb43e9aa69: No bootstrap required, opened a new log
I20260809 22:53:24.313271 29028 ts_tablet_manager.cc:1397] T 277c5f2bf11b4167af28ab8497bdcf9f P b84b90efbbdb40bebe1b5adb43e9aa69: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260809 22:53:24.313836 29028 raft_consensus.cc:348] T 277c5f2bf11b4167af28ab8497bdcf9f P b84b90efbbdb40bebe1b5adb43e9aa69 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b84b90efbbdb40bebe1b5adb43e9aa69" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 42763 } } peers { permanent_uuid: "a85606f73667466f804f9495aab892b4" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 46333 } } peers { permanent_uuid: "ef758642c9de4d939a89e3d634d6a636" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 41323 } }
I20260809 22:53:24.314008 29028 raft_consensus.cc:374] T 277c5f2bf11b4167af28ab8497bdcf9f P b84b90efbbdb40bebe1b5adb43e9aa69 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:53:24.314070 29028 raft_consensus.cc:729] T 277c5f2bf11b4167af28ab8497bdcf9f P b84b90efbbdb40bebe1b5adb43e9aa69 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b84b90efbbdb40bebe1b5adb43e9aa69, State: Initialized, Role: FOLLOWER
I20260809 22:53:24.314265 29028 consensus_queue.cc:260] T 277c5f2bf11b4167af28ab8497bdcf9f P b84b90efbbdb40bebe1b5adb43e9aa69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b84b90efbbdb40bebe1b5adb43e9aa69" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 42763 } } peers { permanent_uuid: "a85606f73667466f804f9495aab892b4" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 46333 } } peers { permanent_uuid: "ef758642c9de4d939a89e3d634d6a636" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 41323 } }
I20260809 22:53:24.314730 29028 ts_tablet_manager.cc:1428] T 277c5f2bf11b4167af28ab8497bdcf9f P b84b90efbbdb40bebe1b5adb43e9aa69: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260809 22:53:24.314776 29031 tablet_bootstrap.cc:654] T 277c5f2bf11b4167af28ab8497bdcf9f P a85606f73667466f804f9495aab892b4: Neither blocks nor log segments found. Creating new log.
I20260809 22:53:24.320132 29029 tablet_bootstrap.cc:492] T 277c5f2bf11b4167af28ab8497bdcf9f P ef758642c9de4d939a89e3d634d6a636: Bootstrap starting.
I20260809 22:53:24.321527 29029 tablet_bootstrap.cc:654] T 277c5f2bf11b4167af28ab8497bdcf9f P ef758642c9de4d939a89e3d634d6a636: Neither blocks nor log segments found. Creating new log.
I20260809 22:53:24.322012 29031 tablet_bootstrap.cc:492] T 277c5f2bf11b4167af28ab8497bdcf9f P a85606f73667466f804f9495aab892b4: No bootstrap required, opened a new log
I20260809 22:53:24.322201 29031 ts_tablet_manager.cc:1397] T 277c5f2bf11b4167af28ab8497bdcf9f P a85606f73667466f804f9495aab892b4: Time spent bootstrapping tablet: real 0.009s user 0.003s sys 0.000s
I20260809 22:53:24.322849 29031 raft_consensus.cc:348] T 277c5f2bf11b4167af28ab8497bdcf9f P a85606f73667466f804f9495aab892b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b84b90efbbdb40bebe1b5adb43e9aa69" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 42763 } } peers { permanent_uuid: "a85606f73667466f804f9495aab892b4" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 46333 } } peers { permanent_uuid: "ef758642c9de4d939a89e3d634d6a636" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 41323 } }
I20260809 22:53:24.323096 29031 raft_consensus.cc:374] T 277c5f2bf11b4167af28ab8497bdcf9f P a85606f73667466f804f9495aab892b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:53:24.323431 29029 tablet_bootstrap.cc:492] T 277c5f2bf11b4167af28ab8497bdcf9f P ef758642c9de4d939a89e3d634d6a636: No bootstrap required, opened a new log
I20260809 22:53:24.323405 29031 raft_consensus.cc:729] T 277c5f2bf11b4167af28ab8497bdcf9f P a85606f73667466f804f9495aab892b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a85606f73667466f804f9495aab892b4, State: Initialized, Role: FOLLOWER
I20260809 22:53:24.323659 29029 ts_tablet_manager.cc:1397] T 277c5f2bf11b4167af28ab8497bdcf9f P ef758642c9de4d939a89e3d634d6a636: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.003s
I20260809 22:53:24.323685 29031 consensus_queue.cc:260] T 277c5f2bf11b4167af28ab8497bdcf9f P a85606f73667466f804f9495aab892b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b84b90efbbdb40bebe1b5adb43e9aa69" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 42763 } } peers { permanent_uuid: "a85606f73667466f804f9495aab892b4" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 46333 } } peers { permanent_uuid: "ef758642c9de4d939a89e3d634d6a636" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 41323 } }
I20260809 22:53:24.324224 29031 ts_tablet_manager.cc:1428] T 277c5f2bf11b4167af28ab8497bdcf9f P a85606f73667466f804f9495aab892b4: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260809 22:53:24.324457 29029 raft_consensus.cc:348] T 277c5f2bf11b4167af28ab8497bdcf9f P ef758642c9de4d939a89e3d634d6a636 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b84b90efbbdb40bebe1b5adb43e9aa69" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 42763 } } peers { permanent_uuid: "a85606f73667466f804f9495aab892b4" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 46333 } } peers { permanent_uuid: "ef758642c9de4d939a89e3d634d6a636" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 41323 } }
I20260809 22:53:24.324709 29029 raft_consensus.cc:374] T 277c5f2bf11b4167af28ab8497bdcf9f P ef758642c9de4d939a89e3d634d6a636 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:53:24.324797 29029 raft_consensus.cc:729] T 277c5f2bf11b4167af28ab8497bdcf9f P ef758642c9de4d939a89e3d634d6a636 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef758642c9de4d939a89e3d634d6a636, State: Initialized, Role: FOLLOWER
I20260809 22:53:24.325071 29029 consensus_queue.cc:260] T 277c5f2bf11b4167af28ab8497bdcf9f P ef758642c9de4d939a89e3d634d6a636 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b84b90efbbdb40bebe1b5adb43e9aa69" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 42763 } } peers { permanent_uuid: "a85606f73667466f804f9495aab892b4" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 46333 } } peers { permanent_uuid: "ef758642c9de4d939a89e3d634d6a636" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 41323 } }
I20260809 22:53:24.325667 29029 ts_tablet_manager.cc:1428] T 277c5f2bf11b4167af28ab8497bdcf9f P ef758642c9de4d939a89e3d634d6a636: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260809 22:53:24.388725 28911 raft_consensus.cc:482] T 277c5f2bf11b4167af28ab8497bdcf9f P b84b90efbbdb40bebe1b5adb43e9aa69 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 22:53:24.388967 28911 raft_consensus.cc:504] T 277c5f2bf11b4167af28ab8497bdcf9f P b84b90efbbdb40bebe1b5adb43e9aa69 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b84b90efbbdb40bebe1b5adb43e9aa69" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 42763 } } peers { permanent_uuid: "a85606f73667466f804f9495aab892b4" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 46333 } } peers { permanent_uuid: "ef758642c9de4d939a89e3d634d6a636" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 41323 } }
I20260809 22:53:24.390472 28911 leader_election.cc:288] T 277c5f2bf11b4167af28ab8497bdcf9f P b84b90efbbdb40bebe1b5adb43e9aa69 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a85606f73667466f804f9495aab892b4 (127.15.113.130:46333), ef758642c9de4d939a89e3d634d6a636 (127.15.113.131:41323)
I20260809 22:53:24.398061 29063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:24.390783 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.131:41323 (local address 127.15.113.129:34663)
0809 22:53:24.391505 (+ 722us) negotiation.cc:107] Waiting for socket to connect
0809 22:53:24.391520 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 22:53:24.391613 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:53:24.392487 (+ 874us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:53:24.392491 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:53:24.392522 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:53:24.392966 (+ 444us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:53:24.392981 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:53:24.394432 (+ 1451us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:53:24.394437 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:53:24.396196 (+ 1759us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:53:24.396207 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:53:24.397322 (+ 1115us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:53:24.397328 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:53:24.397504 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:24.397537 (+ 33us) client_negotiation.cc:769] Sending connection context
0809 22:53:24.397630 (+ 93us) client_negotiation.cc:240] Negotiation successful
0809 22:53:24.397754 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":615,"thread_start_us":881,"threads_started":1}
I20260809 22:53:24.398150 29064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:24.391210 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.113.129:34663 (local address 127.15.113.131:41323)
0809 22:53:24.392044 (+ 834us) server_negotiation.cc:206] Beginning negotiation
0809 22:53:24.392049 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:53:24.392074 (+ 25us) server_negotiation.cc:407] Connection header received
0809 22:53:24.392143 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:53:24.392148 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:53:24.392222 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:53:24.392320 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:53:24.393144 (+ 824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:24.394277 (+ 1133us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:24.396327 (+ 2050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:24.397145 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:24.397293 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:24.397366 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0809 22:53:24.397680 (+ 314us) server_negotiation.cc:299] Negotiation successful
0809 22:53:24.397824 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":702,"thread_start_us":877,"threads_started":1}
I20260809 22:53:24.398767 29060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:24.390598 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.130:46333 (local address 127.15.113.129:34709)
0809 22:53:24.391255 (+ 657us) negotiation.cc:107] Waiting for socket to connect
0809 22:53:24.391274 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 22:53:24.391415 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:53:24.392272 (+ 857us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:53:24.392277 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:53:24.392310 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:53:24.392576 (+ 266us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:53:24.392594 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:53:24.393873 (+ 1279us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:53:24.393878 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:53:24.395761 (+ 1883us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:53:24.395776 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:53:24.397518 (+ 1742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:53:24.397524 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:53:24.397682 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:24.397712 (+ 30us) client_negotiation.cc:769] Sending connection context
0809 22:53:24.398352 (+ 640us) client_negotiation.cc:240] Negotiation successful
0809 22:53:24.398480 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":517,"thread_start_us":370,"threads_started":1}
I20260809 22:53:24.398663 28802 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "277c5f2bf11b4167af28ab8497bdcf9f" candidate_uuid: "b84b90efbbdb40bebe1b5adb43e9aa69" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef758642c9de4d939a89e3d634d6a636" is_pre_election: true
I20260809 22:53:24.398910 28802 raft_consensus.cc:2454] T 277c5f2bf11b4167af28ab8497bdcf9f P ef758642c9de4d939a89e3d634d6a636 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b84b90efbbdb40bebe1b5adb43e9aa69 in term 0.
I20260809 22:53:24.399158 29040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:24.391516 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.113.129:34709 (local address 127.15.113.130:46333)
0809 22:53:24.391697 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0809 22:53:24.391703 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:53:24.391722 (+ 19us) server_negotiation.cc:407] Connection header received
0809 22:53:24.391801 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:53:24.391807 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:53:24.391881 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:53:24.391968 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:53:24.392849 (+ 881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:24.393737 (+ 888us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:24.395932 (+ 2195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:24.397265 (+ 1333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:24.397427 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:24.397499 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0809 22:53:24.397857 (+ 358us) server_negotiation.cc:299] Negotiation successful
0809 22:53:24.397988 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260809 22:53:24.399452 28261 leader_election.cc:302] T 277c5f2bf11b4167af28ab8497bdcf9f P b84b90efbbdb40bebe1b5adb43e9aa69 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b84b90efbbdb40bebe1b5adb43e9aa69, ef758642c9de4d939a89e3d634d6a636; no voters:
I20260809 22:53:24.399770 28594 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "277c5f2bf11b4167af28ab8497bdcf9f" candidate_uuid: "b84b90efbbdb40bebe1b5adb43e9aa69" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a85606f73667466f804f9495aab892b4" is_pre_election: true
I20260809 22:53:24.399873 28911 raft_consensus.cc:2781] T 277c5f2bf11b4167af28ab8497bdcf9f P b84b90efbbdb40bebe1b5adb43e9aa69 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 22:53:24.399986 28911 raft_consensus.cc:482] T 277c5f2bf11b4167af28ab8497bdcf9f P b84b90efbbdb40bebe1b5adb43e9aa69 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 22:53:24.400035 28594 raft_consensus.cc:2454] T 277c5f2bf11b4167af28ab8497bdcf9f P a85606f73667466f804f9495aab892b4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b84b90efbbdb40bebe1b5adb43e9aa69 in term 0.
I20260809 22:53:24.400084 28911 raft_consensus.cc:3037] T 277c5f2bf11b4167af28ab8497bdcf9f P b84b90efbbdb40bebe1b5adb43e9aa69 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:53:24.402372 28911 raft_consensus.cc:504] T 277c5f2bf11b4167af28ab8497bdcf9f P b84b90efbbdb40bebe1b5adb43e9aa69 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b84b90efbbdb40bebe1b5adb43e9aa69" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 42763 } } peers { permanent_uuid: "a85606f73667466f804f9495aab892b4" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 46333 } } peers { permanent_uuid: "ef758642c9de4d939a89e3d634d6a636" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 41323 } }
I20260809 22:53:24.402897 28911 leader_election.cc:288] T 277c5f2bf11b4167af28ab8497bdcf9f P b84b90efbbdb40bebe1b5adb43e9aa69 [CANDIDATE]: Term 1 election: Requested vote from peers a85606f73667466f804f9495aab892b4 (127.15.113.130:46333), ef758642c9de4d939a89e3d634d6a636 (127.15.113.131:41323)
I20260809 22:53:24.403338 28594 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "277c5f2bf11b4167af28ab8497bdcf9f" candidate_uuid: "b84b90efbbdb40bebe1b5adb43e9aa69" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a85606f73667466f804f9495aab892b4"
I20260809 22:53:24.403463 28802 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "277c5f2bf11b4167af28ab8497bdcf9f" candidate_uuid: "b84b90efbbdb40bebe1b5adb43e9aa69" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef758642c9de4d939a89e3d634d6a636"
I20260809 22:53:24.403548 28594 raft_consensus.cc:3037] T 277c5f2bf11b4167af28ab8497bdcf9f P a85606f73667466f804f9495aab892b4 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:53:24.403664 28802 raft_consensus.cc:3037] T 277c5f2bf11b4167af28ab8497bdcf9f P ef758642c9de4d939a89e3d634d6a636 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:53:24.406424 28802 raft_consensus.cc:2454] T 277c5f2bf11b4167af28ab8497bdcf9f P ef758642c9de4d939a89e3d634d6a636 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b84b90efbbdb40bebe1b5adb43e9aa69 in term 1.
I20260809 22:53:24.406553 28594 raft_consensus.cc:2454] T 277c5f2bf11b4167af28ab8497bdcf9f P a85606f73667466f804f9495aab892b4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b84b90efbbdb40bebe1b5adb43e9aa69 in term 1.
I20260809 22:53:24.406821 28261 leader_election.cc:302] T 277c5f2bf11b4167af28ab8497bdcf9f P b84b90efbbdb40bebe1b5adb43e9aa69 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b84b90efbbdb40bebe1b5adb43e9aa69, ef758642c9de4d939a89e3d634d6a636; no voters:
I20260809 22:53:24.407070 28911 raft_consensus.cc:2781] T 277c5f2bf11b4167af28ab8497bdcf9f P b84b90efbbdb40bebe1b5adb43e9aa69 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 22:53:24.407511 28911 raft_consensus.cc:686] T 277c5f2bf11b4167af28ab8497bdcf9f P b84b90efbbdb40bebe1b5adb43e9aa69 [term 1 LEADER]: Becoming Leader. State: Replica: b84b90efbbdb40bebe1b5adb43e9aa69, State: Running, Role: LEADER
I20260809 22:53:24.407897 28911 consensus_queue.cc:237] T 277c5f2bf11b4167af28ab8497bdcf9f P b84b90efbbdb40bebe1b5adb43e9aa69 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b84b90efbbdb40bebe1b5adb43e9aa69" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 42763 } } peers { permanent_uuid: "a85606f73667466f804f9495aab892b4" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 46333 } } peers { permanent_uuid: "ef758642c9de4d939a89e3d634d6a636" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 41323 } }
I20260809 22:53:24.411303 28128 catalog_manager.cc:5495] T 277c5f2bf11b4167af28ab8497bdcf9f P b84b90efbbdb40bebe1b5adb43e9aa69 reported cstate change: term changed from 0 to 1, leader changed from <none> to b84b90efbbdb40bebe1b5adb43e9aa69 (127.15.113.129). New cstate: current_term: 1 leader_uuid: "b84b90efbbdb40bebe1b5adb43e9aa69" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b84b90efbbdb40bebe1b5adb43e9aa69" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 42763 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a85606f73667466f804f9495aab892b4" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 46333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ef758642c9de4d939a89e3d634d6a636" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 41323 } health_report { overall_health: UNKNOWN } } }
I20260809 22:53:24.455452 29071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:24.448659 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35600 (local address 127.15.113.129:42763)
0809 22:53:24.449184 (+ 525us) server_negotiation.cc:206] Beginning negotiation
0809 22:53:24.449188 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 22:53:24.449231 (+ 43us) server_negotiation.cc:407] Connection header received
0809 22:53:24.449438 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:53:24.449441 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:53:24.449526 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:53:24.449678 (+ 152us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 22:53:24.450062 (+ 384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:24.450979 (+ 917us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:24.452860 (+ 1881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:24.453326 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:24.453416 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:24.454009 (+ 593us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 22:53:24.454796 (+ 787us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 22:53:24.454946 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0809 22:53:24.455008 (+ 62us) server_negotiation.cc:299] Negotiation successful
0809 22:53:24.455088 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":419,"thread_start_us":281,"threads_started":1}
I20260809 22:53:24.470453 29040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:24.465178 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.113.129:39911 (local address 127.15.113.130:46333)
0809 22:53:24.465324 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0809 22:53:24.465329 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:53:24.465932 (+ 603us) server_negotiation.cc:407] Connection header received
0809 22:53:24.466107 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:53:24.466111 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:53:24.466215 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:53:24.466293 (+ 78us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 22:53:24.466754 (+ 461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:24.467506 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:24.468797 (+ 1291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:24.469268 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:24.469381 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:24.469767 (+ 386us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 22:53:24.469864 (+ 97us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 22:53:24.469935 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0809 22:53:24.470068 (+ 133us) server_negotiation.cc:299] Negotiation successful
0809 22:53:24.470149 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260809 22:53:24.470521 29074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:24.465270 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.130:46333 (local address 127.15.113.129:39911)
0809 22:53:24.465838 (+ 568us) negotiation.cc:107] Waiting for socket to connect
0809 22:53:24.465853 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 22:53:24.465978 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:53:24.466320 (+ 342us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:53:24.466325 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:53:24.466336 (+ 11us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 22:53:24.466619 (+ 283us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:53:24.466628 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:53:24.467647 (+ 1019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:53:24.467652 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:53:24.468666 (+ 1014us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:53:24.468677 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:53:24.469435 (+ 758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:53:24.469441 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:53:24.469555 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:24.469609 (+ 54us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 22:53:24.469957 (+ 348us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 22:53:24.469968 (+ 11us) client_negotiation.cc:769] Sending connection context
0809 22:53:24.470033 (+ 65us) client_negotiation.cc:240] Negotiation successful
0809 22:53:24.470169 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":438,"thread_start_us":287,"threads_started":1}
I20260809 22:53:24.479079 28802 raft_consensus.cc:1261] T 277c5f2bf11b4167af28ab8497bdcf9f P ef758642c9de4d939a89e3d634d6a636 [term 1 FOLLOWER]: Refusing update from remote peer b84b90efbbdb40bebe1b5adb43e9aa69: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 22:53:24.479234 28594 raft_consensus.cc:1261] T 277c5f2bf11b4167af28ab8497bdcf9f P a85606f73667466f804f9495aab892b4 [term 1 FOLLOWER]: Refusing update from remote peer b84b90efbbdb40bebe1b5adb43e9aa69: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 22:53:24.479723 28911 consensus_queue.cc:1035] T 277c5f2bf11b4167af28ab8497bdcf9f P b84b90efbbdb40bebe1b5adb43e9aa69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef758642c9de4d939a89e3d634d6a636" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 41323 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 22:53:24.480312 28911 consensus_queue.cc:1035] T 277c5f2bf11b4167af28ab8497bdcf9f P b84b90efbbdb40bebe1b5adb43e9aa69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a85606f73667466f804f9495aab892b4" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 46333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 22:53:24.531931 29071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:24.524044 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.113.130:60239 (local address 127.15.113.129:42763)
0809 22:53:24.524224 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0809 22:53:24.524230 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:53:24.526140 (+ 1910us) server_negotiation.cc:407] Connection header received
0809 22:53:24.526333 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:53:24.526339 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:53:24.526415 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:53:24.526502 (+ 87us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 22:53:24.527054 (+ 552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:24.528138 (+ 1084us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:24.530298 (+ 2160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:24.530716 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:24.530823 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:24.531263 (+ 440us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 22:53:24.531365 (+ 102us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 22:53:24.531436 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0809 22:53:24.531590 (+ 154us) server_negotiation.cc:299] Negotiation successful
0809 22:53:24.531674 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260809 22:53:24.532168 29080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:24.523491 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.129:42763 (local address 127.15.113.130:60239)
0809 22:53:24.526034 (+ 2543us) negotiation.cc:107] Waiting for socket to connect
0809 22:53:24.526053 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 22:53:24.526205 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:53:24.526546 (+ 341us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:53:24.526551 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:53:24.526563 (+ 12us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 22:53:24.526907 (+ 344us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:53:24.526919 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:53:24.528363 (+ 1444us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:53:24.528368 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:53:24.530156 (+ 1788us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:53:24.530170 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:53:24.530902 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:53:24.530912 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:53:24.531057 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:24.531096 (+ 39us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 22:53:24.531468 (+ 372us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 22:53:24.531480 (+ 12us) client_negotiation.cc:769] Sending connection context
0809 22:53:24.531555 (+ 75us) client_negotiation.cc:240] Negotiation successful
0809 22:53:24.531689 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2372,"thread_start_us":2213,"threads_started":1}
I20260809 22:53:24.567274 28128 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49160:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 22:53:24.567577 28128 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49160:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 22:53:24.571144 28128 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f37bab48ada649e6bd170b082f14874f: Sending DeleteTablet for 3 replicas of tablet 277c5f2bf11b4167af28ab8497bdcf9f
I20260809 22:53:24.572134 28765 tablet_service.cc:1514] Processing DeleteTablet for tablet 277c5f2bf11b4167af28ab8497bdcf9f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:53:24 UTC) from {username='oryx', principal='oryx/127.15.113.190@KRBTEST.COM'} at 127.0.0.1:49762
I20260809 22:53:24.572440 29009 tablet_replica.cc:330] T 277c5f2bf11b4167af28ab8497bdcf9f P ef758642c9de4d939a89e3d634d6a636: stopping tablet replica
I20260809 22:53:24.572472 28330 tablet_service.cc:1514] Processing DeleteTablet for tablet 277c5f2bf11b4167af28ab8497bdcf9f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:53:24 UTC) from {username='oryx', principal='oryx/127.15.113.190@KRBTEST.COM'} at 127.0.0.1:35596
I20260809 22:53:24.572623 29009 raft_consensus.cc:2229] T 277c5f2bf11b4167af28ab8497bdcf9f P ef758642c9de4d939a89e3d634d6a636 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 22:53:24.572845 29009 raft_consensus.cc:2258] T 277c5f2bf11b4167af28ab8497bdcf9f P ef758642c9de4d939a89e3d634d6a636 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 22:53:24.573171 29006 tablet_replica.cc:330] T 277c5f2bf11b4167af28ab8497bdcf9f P b84b90efbbdb40bebe1b5adb43e9aa69: stopping tablet replica
I20260809 22:53:24.573354 29006 raft_consensus.cc:2229] T 277c5f2bf11b4167af28ab8497bdcf9f P b84b90efbbdb40bebe1b5adb43e9aa69 [term 1 LEADER]: Raft consensus shutting down.
I20260809 22:53:24.573866 29006 raft_consensus.cc:2258] T 277c5f2bf11b4167af28ab8497bdcf9f P b84b90efbbdb40bebe1b5adb43e9aa69 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 22:53:24.574394 28549 tablet_service.cc:1514] Processing DeleteTablet for tablet 277c5f2bf11b4167af28ab8497bdcf9f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:53:24 UTC) from {username='oryx', principal='oryx/127.15.113.190@KRBTEST.COM'} at 127.0.0.1:37298
I20260809 22:53:24.574698 29007 tablet_replica.cc:330] T 277c5f2bf11b4167af28ab8497bdcf9f P a85606f73667466f804f9495aab892b4: stopping tablet replica
I20260809 22:53:24.574880 29007 raft_consensus.cc:2229] T 277c5f2bf11b4167af28ab8497bdcf9f P a85606f73667466f804f9495aab892b4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 22:53:24.575085 29007 raft_consensus.cc:2258] T 277c5f2bf11b4167af28ab8497bdcf9f P a85606f73667466f804f9495aab892b4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 22:53:24.575219 29006 ts_tablet_manager.cc:1905] T 277c5f2bf11b4167af28ab8497bdcf9f P b84b90efbbdb40bebe1b5adb43e9aa69: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 22:53:24.576377 29007 ts_tablet_manager.cc:1905] T 277c5f2bf11b4167af28ab8497bdcf9f P a85606f73667466f804f9495aab892b4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 22:53:24.581859 29009 ts_tablet_manager.cc:1905] T 277c5f2bf11b4167af28ab8497bdcf9f P ef758642c9de4d939a89e3d634d6a636: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 22:53:24.582752 29006 ts_tablet_manager.cc:1918] T 277c5f2bf11b4167af28ab8497bdcf9f P b84b90efbbdb40bebe1b5adb43e9aa69: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 22:53:24.582887 29006 log.cc:1196] T 277c5f2bf11b4167af28ab8497bdcf9f P b84b90efbbdb40bebe1b5adb43e9aa69: Deleting WAL directory at /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315935617919-15814-0/minicluster-data/ts-0/wal/wals/277c5f2bf11b4167af28ab8497bdcf9f
I20260809 22:53:24.583463 29006 ts_tablet_manager.cc:1939] T 277c5f2bf11b4167af28ab8497bdcf9f P b84b90efbbdb40bebe1b5adb43e9aa69: Deleting consensus metadata
I20260809 22:53:24.584637 28102 catalog_manager.cc:4841] TS b84b90efbbdb40bebe1b5adb43e9aa69 (127.15.113.129:42763): tablet 277c5f2bf11b4167af28ab8497bdcf9f (table test-table [id=30de39925fb448b48ee0ef0c28c37779]) successfully deleted
I20260809 22:53:24.588685 29009 ts_tablet_manager.cc:1918] T 277c5f2bf11b4167af28ab8497bdcf9f P ef758642c9de4d939a89e3d634d6a636: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 22:53:24.588824 29009 log.cc:1196] T 277c5f2bf11b4167af28ab8497bdcf9f P ef758642c9de4d939a89e3d634d6a636: Deleting WAL directory at /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315935617919-15814-0/minicluster-data/ts-2/wal/wals/277c5f2bf11b4167af28ab8497bdcf9f
I20260809 22:53:24.589947 15814 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskItYPJR/build/asan/bin/kudu pid 28236
I20260809 22:53:24.595396 29007 ts_tablet_manager.cc:1918] T 277c5f2bf11b4167af28ab8497bdcf9f P a85606f73667466f804f9495aab892b4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 22:53:24.595547 29007 log.cc:1196] T 277c5f2bf11b4167af28ab8497bdcf9f P a85606f73667466f804f9495aab892b4: Deleting WAL directory at /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315935617919-15814-0/minicluster-data/ts-1/wal/wals/277c5f2bf11b4167af28ab8497bdcf9f
I20260809 22:53:24.596045 29007 ts_tablet_manager.cc:1939] T 277c5f2bf11b4167af28ab8497bdcf9f P a85606f73667466f804f9495aab892b4: Deleting consensus metadata
I20260809 22:53:24.597229 28103 catalog_manager.cc:4841] TS a85606f73667466f804f9495aab892b4 (127.15.113.130:46333): tablet 277c5f2bf11b4167af28ab8497bdcf9f (table test-table [id=30de39925fb448b48ee0ef0c28c37779]) successfully deleted
I20260809 22:53:24.597851 29009 ts_tablet_manager.cc:1939] T 277c5f2bf11b4167af28ab8497bdcf9f P ef758642c9de4d939a89e3d634d6a636: Deleting consensus metadata
I20260809 22:53:24.598930 28106 catalog_manager.cc:4841] TS ef758642c9de4d939a89e3d634d6a636 (127.15.113.131:41323): tablet 277c5f2bf11b4167af28ab8497bdcf9f (table test-table [id=30de39925fb448b48ee0ef0c28c37779]) successfully deleted
Aug 09 22:53:24 dist-test-slave-ncp9 krb5kdc[28066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316003, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.15.113.129@KRBTEST.COM
I20260809 22:53:24.620239 29071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:24.591280 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35602 (local address 127.15.113.129:42763)
0809 22:53:24.591728 (+ 448us) server_negotiation.cc:206] Beginning negotiation
0809 22:53:24.591735 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:53:24.601431 (+ 9696us) server_negotiation.cc:407] Connection header received
0809 22:53:24.601615 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:53:24.601620 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:53:24.601738 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:53:24.601864 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:53:24.603372 (+ 1508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:24.604549 (+ 1177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:24.606409 (+ 1860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:24.607158 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:24.607301 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:24.612594 (+ 5293us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:53:24.612609 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:53:24.612629 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:53:24.612683 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:53:24.616873 (+ 4190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:53:24.617789 (+ 916us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:53:24.617798 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:53:24.617802 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:53:24.617968 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:53:24.618431 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:53:24.618436 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:53:24.618440 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:53:24.618906 (+ 466us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:53:24.619262 (+ 356us) server_negotiation.cc:1035] Waiting for connection context
0809 22:53:24.619595 (+ 333us) server_negotiation.cc:299] Negotiation successful
0809 22:53:24.619844 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":285}
I20260809 22:53:24.621070 28432 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:53:24.729522 15814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskItYPJR/build/asan/bin/kudu with pid 28236
I20260809 22:53:24.757979 15814 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskItYPJR/build/asan/bin/kudu pid 28443
Aug 09 22:53:24 dist-test-slave-ncp9 krb5kdc[28066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316003, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.15.113.130@KRBTEST.COM
I20260809 22:53:24.771955 29040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:24.758766 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37320 (local address 127.15.113.130:46333)
0809 22:53:24.758937 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0809 22:53:24.758942 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:53:24.758961 (+ 19us) server_negotiation.cc:407] Connection header received
0809 22:53:24.759025 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:53:24.759029 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:53:24.759090 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:53:24.759173 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:53:24.760414 (+ 1241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:24.761167 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:24.762418 (+ 1251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:24.762866 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:24.762975 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:24.766848 (+ 3873us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:53:24.766862 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:53:24.766879 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:53:24.766917 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:53:24.769436 (+ 2519us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:53:24.770175 (+ 739us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:53:24.770182 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:53:24.770184 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:53:24.770301 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:53:24.770631 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:53:24.770634 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:53:24.770636 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:53:24.771030 (+ 394us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:53:24.771238 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0809 22:53:24.771546 (+ 308us) server_negotiation.cc:299] Negotiation successful
0809 22:53:24.771719 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260809 22:53:24.772616 28655 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:53:24.915082 15814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskItYPJR/build/asan/bin/kudu with pid 28443
I20260809 22:53:24.945607 15814 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskItYPJR/build/asan/bin/kudu pid 28666
Aug 09 22:53:24 dist-test-slave-ncp9 krb5kdc[28066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316003, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.15.113.131@KRBTEST.COM
I20260809 22:53:24.960074 29092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:24.946310 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49780 (local address 127.15.113.131:41323)
0809 22:53:24.946736 (+ 426us) server_negotiation.cc:206] Beginning negotiation
0809 22:53:24.946741 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:53:24.946757 (+ 16us) server_negotiation.cc:407] Connection header received
0809 22:53:24.946816 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:53:24.946819 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:53:24.946889 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:53:24.946979 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:53:24.948211 (+ 1232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:24.948880 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:24.950141 (+ 1261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:24.950559 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:24.950645 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:24.954738 (+ 4093us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:53:24.954754 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:53:24.954772 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:53:24.954826 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:53:24.957449 (+ 2623us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:53:24.958072 (+ 623us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:53:24.958079 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:53:24.958081 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:53:24.958256 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:53:24.958630 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:53:24.958634 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:53:24.958636 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:53:24.958983 (+ 347us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:53:24.959290 (+ 307us) server_negotiation.cc:1035] Waiting for connection context
0809 22:53:24.959562 (+ 272us) server_negotiation.cc:299] Negotiation successful
0809 22:53:24.959782 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":328,"thread_start_us":215,"threads_started":1}
I20260809 22:53:24.960667 28857 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:53:25.080862 15814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskItYPJR/build/asan/bin/kudu with pid 28666
I20260809 22:53:25.105073 15814 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskItYPJR/build/asan/bin/kudu pid 28082
I20260809 22:53:25.106231 28189 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:53:25.220573 15814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskItYPJR/build/asan/bin/kudu with pid 28082
2026-08-09T22:53:25Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (6322 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786315935617919-15814-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 22:53:25 dist-test-slave-ncp9 krb5kdc[29100](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 22:53:25 dist-test-slave-ncp9 krb5kdc[29100](info): set up 2 sockets
Aug 09 22:53:25 dist-test-slave-ncp9 krb5kdc[29100](info): commencing operation
krb5kdc: starting...
W20260809 22:53:26.422335 15814 mini_kdc.cc:121] Time spent starting KDC: real 1.149s user 0.000s sys 0.032s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786315935617919-15814-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786315935617919-15814-0/krb5kdc/test-user.keytab.
Aug 09 22:53:26 dist-test-slave-ncp9 krb5kdc[29100](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316006, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T22:53:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T22:53:26Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.113.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.113.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.113.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.190.keytab.
Entry for principal kudu/127.15.113.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.190.keytab.
I20260809 22:53:26.666445 15814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskItYPJR/build/asan/bin/kudu
/tmp/dist-test-taskItYPJR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786315935617919-15814-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786315935617919-15814-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786315935617919-15814-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786315935617919-15814-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.113.190:43673
--webserver_interface=127.15.113.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.190.keytab
--principal=kudu/127.15.113.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.113.148:40223
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.113.190:43673
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786315935617919-15814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786315935617919-15814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786315935617919-15814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:53:26.996170 29118 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:53:26.996486 29118 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:53:26.996641 29118 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:53:27.003841 29118 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 22:53:27.003959 29118 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:53:27.004048 29118 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:53:27.004132 29118 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 22:53:27.004212 29118 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 22:53:27.013797 29118 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.113.148:40223
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786315935617919-15814-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786315935617919-15814-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.113.190:43673
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.190.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.113.190:43673
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786315935617919-15814-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.113.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786315935617919-15814-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:53:27.015031 29118 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:53:27.016338 29118 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:53:27.027838 29123 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:53:27.027885 29124 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:53:27.027909 29127 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:53:27.029083 29118 server_base.cc:1034] running on GCE node
I20260809 22:53:27.029827 29118 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:53:27.031143 29118 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:53:27.032361 29118 hybrid_clock.cc:648] HybridClock initialized: now 1786316007032306 us; error 35 us; skew 500 ppm
I20260809 22:53:27.032693 29118 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-08-09T22:53:27Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1874 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315935617919-15814-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 22:53:27 dist-test-slave-ncp9 krb5kdc[29136](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 22:53:27 dist-test-slave-ncp9 krb5kdc[29136](info): set up 2 sockets
Aug 09 22:53:27 dist-test-slave-ncp9 krb5kdc[29136](info): commencing operation
krb5kdc: starting...
W20260809 22:53:28.304602 15814 mini_kdc.cc:121] Time spent starting KDC: real 1.157s user 0.000s sys 0.033s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315935617919-15814-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315935617919-15814-0/krb5kdc/test-user.keytab.
Aug 09 22:53:28 dist-test-slave-ncp9 krb5kdc[29136](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316008, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T22:53:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T22:53:28Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.113.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.113.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.113.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.190.keytab.
Entry for principal kudu/127.15.113.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.190.keytab.
I20260809 22:53:28.546511 15814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskItYPJR/build/asan/bin/kudu
/tmp/dist-test-taskItYPJR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315935617919-15814-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315935617919-15814-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315935617919-15814-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315935617919-15814-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.113.190:44011
--webserver_interface=127.15.113.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.190.keytab
--principal=kudu/127.15.113.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.113.148:40757
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.113.190:44011
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315935617919-15814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315935617919-15814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315935617919-15814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:53:28.870888 29152 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:53:28.871179 29152 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:53:28.871310 29152 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:53:28.878322 29152 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 22:53:28.878429 29152 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:53:28.878546 29152 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:53:28.878633 29152 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 22:53:28.878698 29152 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 22:53:28.888466 29152 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.113.148:40757
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315935617919-15814-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315935617919-15814-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.113.190:44011
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.190.keytab
--principal=kudu/127.15.113.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.113.190:44011
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315935617919-15814-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.113.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315935617919-15814-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:53:28.889768 29152 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:53:28.891041 29152 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:53:28.902324 29160 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:53:28.902375 29161 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:53:28.902348 29165 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:53:28.903296 29152 server_base.cc:1034] running on GCE node
I20260809 22:53:28.903987 29152 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:53:28.905340 29152 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:53:28.906543 29152 hybrid_clock.cc:648] HybridClock initialized: now 1786316008906500 us; error 58 us; skew 500 ppm
I20260809 22:53:28.906852 29152 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:53:28 dist-test-slave-ncp9 krb5kdc[29136](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316008, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.113.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:53:28.915156 29152 init.cc:377] Logged in from keytab as kudu/127.15.113.190@KRBTEST.COM (short username kudu)
I20260809 22:53:28.916546 29152 webserver.cc:460] Webserver started at http://127.15.113.190:34253/ using document root <none> and password file <none>
I20260809 22:53:28.917276 29152 fs_manager.cc:359] Metadata directory not provided
I20260809 22:53:28.917397 29152 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:53:28.917761 29152 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:53:28.920639 29152 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315935617919-15814-0/minicluster-data/master-0/data/instance:
uuid: "2ea0d8ef17c6474690168da53ac2015c"
format_stamp: "Formatted at 2026-08-09 22:53:28 on dist-test-slave-ncp9"
server_key: "4c979b2d913f16758cf23e4ee1af2901"
server_key_iv: "ed80cae4303660da1b26ae342fa5ed26"
server_key_version: "encryptionkey@0"
I20260809 22:53:28.921422 29152 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315935617919-15814-0/minicluster-data/master-0/wal/instance:
uuid: "2ea0d8ef17c6474690168da53ac2015c"
format_stamp: "Formatted at 2026-08-09 22:53:28 on dist-test-slave-ncp9"
server_key: "4c979b2d913f16758cf23e4ee1af2901"
server_key_iv: "ed80cae4303660da1b26ae342fa5ed26"
server_key_version: "encryptionkey@0"
I20260809 22:53:28.926383 29152 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260809 22:53:28.929659 29180 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:53:28.930893 29152 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 22:53:28.931051 29152 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315935617919-15814-0/minicluster-data/master-0/data,/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315935617919-15814-0/minicluster-data/master-0/wal
uuid: "2ea0d8ef17c6474690168da53ac2015c"
format_stamp: "Formatted at 2026-08-09 22:53:28 on dist-test-slave-ncp9"
server_key: "4c979b2d913f16758cf23e4ee1af2901"
server_key_iv: "ed80cae4303660da1b26ae342fa5ed26"
server_key_version: "encryptionkey@0"
I20260809 22:53:28.931236 29152 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315935617919-15814-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315935617919-15814-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315935617919-15814-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:53:28.947872 29152 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:53:28.951479 29152 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:53:28.951753 29152 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:53:28.969105 29152 rpc_server.cc:307] RPC server started. Bound to: 127.15.113.190:44011
I20260809 22:53:28.969115 29254 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.113.190:44011 every 8 connection(s)
I20260809 22:53:28.970464 29152 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315935617919-15814-0/minicluster-data/master-0/data/info.pb
I20260809 22:53:28.974130 29255 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:53:28.979113 15814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskItYPJR/build/asan/bin/kudu as pid 29152
I20260809 22:53:28.979342 15814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315935617919-15814-0/minicluster-data/master-0/wal/instance
I20260809 22:53:28.980176 15814 external_mini_cluster.cc:1428] Setting key 66bdb107bb153c5fa6d81464cb85032b
I20260809 22:53:28.985328 29255 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2ea0d8ef17c6474690168da53ac2015c: Bootstrap starting.
I20260809 22:53:28.988835 29255 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2ea0d8ef17c6474690168da53ac2015c: Neither blocks nor log segments found. Creating new log.
I20260809 22:53:28.990072 29255 log.cc:824] T 00000000000000000000000000000000 P 2ea0d8ef17c6474690168da53ac2015c: Log is configured to *not* fsync() on all Append() calls
I20260809 22:53:28.993363 29255 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2ea0d8ef17c6474690168da53ac2015c: No bootstrap required, opened a new log
Aug 09 22:53:28 dist-test-slave-ncp9 krb5kdc[29136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316008, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.113.190@KRBTEST.COM
I20260809 22:53:29.001224 29255 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2ea0d8ef17c6474690168da53ac2015c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ea0d8ef17c6474690168da53ac2015c" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 44011 } }
I20260809 22:53:29.001583 29255 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2ea0d8ef17c6474690168da53ac2015c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:53:29.001688 29255 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2ea0d8ef17c6474690168da53ac2015c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ea0d8ef17c6474690168da53ac2015c, State: Initialized, Role: FOLLOWER
I20260809 22:53:29.002240 29255 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2ea0d8ef17c6474690168da53ac2015c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ea0d8ef17c6474690168da53ac2015c" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 44011 } }
I20260809 22:53:29.002477 29255 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2ea0d8ef17c6474690168da53ac2015c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 22:53:29.002638 29255 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2ea0d8ef17c6474690168da53ac2015c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 22:53:29.002815 29255 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2ea0d8ef17c6474690168da53ac2015c [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:53:29.003237 29262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:28.982305 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38928 (local address 127.15.113.190:44011)
0809 22:53:28.983163 (+ 858us) server_negotiation.cc:206] Beginning negotiation
0809 22:53:28.983176 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 22:53:28.983216 (+ 40us) server_negotiation.cc:407] Connection header received
0809 22:53:28.984950 (+ 1734us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:53:28.984975 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:53:28.985345 (+ 370us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:53:28.985693 (+ 348us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:53:28.987472 (+ 1779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:28.988738 (+ 1266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:28.990426 (+ 1688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:28.991460 (+ 1034us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:28.991589 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:28.995978 (+ 4389us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:53:28.995988 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:53:28.996002 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:53:28.996041 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:53:28.999015 (+ 2974us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:53:28.999683 (+ 668us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:53:28.999689 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:53:28.999699 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:53:28.999865 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:53:29.000248 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:53:29.000251 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:53:29.000254 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:53:29.000778 (+ 524us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:53:29.001106 (+ 328us) server_negotiation.cc:1035] Waiting for connection context
0809 22:53:29.001526 (+ 420us) server_negotiation.cc:299] Negotiation successful
0809 22:53:29.002131 (+ 605us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":633,"thread_start_us":385,"threads_started":1}
I20260809 22:53:29.005117 29255 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2ea0d8ef17c6474690168da53ac2015c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ea0d8ef17c6474690168da53ac2015c" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 44011 } }
I20260809 22:53:29.005597 29255 leader_election.cc:302] T 00000000000000000000000000000000 P 2ea0d8ef17c6474690168da53ac2015c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2ea0d8ef17c6474690168da53ac2015c; no voters:
I20260809 22:53:29.006158 29255 leader_election.cc:288] T 00000000000000000000000000000000 P 2ea0d8ef17c6474690168da53ac2015c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 22:53:29.006261 29266 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2ea0d8ef17c6474690168da53ac2015c [term 1 FOLLOWER]: Leader election won for term 1
I20260809 22:53:29.006886 29266 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2ea0d8ef17c6474690168da53ac2015c [term 1 LEADER]: Becoming Leader. State: Replica: 2ea0d8ef17c6474690168da53ac2015c, State: Running, Role: LEADER
I20260809 22:53:29.007423 29266 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2ea0d8ef17c6474690168da53ac2015c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ea0d8ef17c6474690168da53ac2015c" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 44011 } }
I20260809 22:53:29.008236 29255 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2ea0d8ef17c6474690168da53ac2015c [sys.catalog]: configured and running, proceeding with master startup.
I20260809 22:53:29.010704 29271 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2ea0d8ef17c6474690168da53ac2015c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2ea0d8ef17c6474690168da53ac2015c. Latest consensus state: current_term: 1 leader_uuid: "2ea0d8ef17c6474690168da53ac2015c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ea0d8ef17c6474690168da53ac2015c" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 44011 } } }
I20260809 22:53:29.011299 29271 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2ea0d8ef17c6474690168da53ac2015c [sys.catalog]: This master's current role is: LEADER
I20260809 22:53:29.010852 29269 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2ea0d8ef17c6474690168da53ac2015c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2ea0d8ef17c6474690168da53ac2015c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ea0d8ef17c6474690168da53ac2015c" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 44011 } } }
I20260809 22:53:29.012629 29269 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2ea0d8ef17c6474690168da53ac2015c [sys.catalog]: This master's current role is: LEADER
I20260809 22:53:29.017657 29282 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 22:53:29.022378 29282 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 22:53:29.030261 29282 catalog_manager.cc:1347] Generated new cluster ID: e087c41c879541c99dcb01859414f64e
I20260809 22:53:29.030429 29282 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 22:53:29.039314 29282 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 22:53:29.041085 29282 catalog_manager.cc:1504] Loading token signing keys...
I20260809 22:53:29.058561 29282 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2ea0d8ef17c6474690168da53ac2015c: Generated new TSK 0
I20260809 22:53:29.059656 29282 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.113.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.113.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.113.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.129.keytab.
Entry for principal kudu/127.15.113.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.129.keytab.
I20260809 22:53:29.126186 15814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskItYPJR/build/asan/bin/kudu
/tmp/dist-test-taskItYPJR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315935617919-15814-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315935617919-15814-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315935617919-15814-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315935617919-15814-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.113.129:0
--local_ip_for_outbound_sockets=127.15.113.129
--webserver_interface=127.15.113.129
--webserver_port=0
--tserver_master_addrs=127.15.113.190:44011
--keytab_file=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.129.keytab
--principal=kudu/127.15.113.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.113.148:40757
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315935617919-15814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315935617919-15814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315935617919-15814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:53:29.454468 29299 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:53:29.454741 29299 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:53:29.454887 29299 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:53:29.461951 29299 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:53:29.462065 29299 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:53:29.462236 29299 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.113.129
I20260809 22:53:29.471647 29299 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.113.148:40757
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315935617919-15814-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315935617919-15814-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.129.keytab
--principal=kudu/127.15.113.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.113.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315935617919-15814-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.113.129
--webserver_port=0
--tserver_master_addrs=127.15.113.190:44011
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.113.129
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315935617919-15814-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:53:29.472822 29299 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:53:29.474097 29299 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:53:29.486577 29305 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:53:29.486490 29308 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:53:29.486538 29306 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:53:29.487413 29299 server_base.cc:1034] running on GCE node
I20260809 22:53:29.487920 29299 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:53:29.488740 29299 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:53:29.489934 29299 hybrid_clock.cc:648] HybridClock initialized: now 1786316009489879 us; error 44 us; skew 500 ppm
I20260809 22:53:29.490240 29299 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:53:29 dist-test-slave-ncp9 krb5kdc[29136](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316009, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.113.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:53:29.493281 29299 init.cc:377] Logged in from keytab as kudu/127.15.113.129@KRBTEST.COM (short username kudu)
I20260809 22:53:29.494760 29299 webserver.cc:460] Webserver started at http://127.15.113.129:35445/ using document root <none> and password file <none>
I20260809 22:53:29.495409 29299 fs_manager.cc:359] Metadata directory not provided
I20260809 22:53:29.495510 29299 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:53:29.495764 29299 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:53:29.498728 29299 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315935617919-15814-0/minicluster-data/ts-0/data/instance:
uuid: "9820493cc168460eaeea066a77cccd4d"
format_stamp: "Formatted at 2026-08-09 22:53:29 on dist-test-slave-ncp9"
server_key: "29926b41f9ec70edcccb0e259c7f4505"
server_key_iv: "71749821fbafc4ef52cb9ab81b1a4df9"
server_key_version: "encryptionkey@0"
I20260809 22:53:29.499446 29299 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315935617919-15814-0/minicluster-data/ts-0/wal/instance:
uuid: "9820493cc168460eaeea066a77cccd4d"
format_stamp: "Formatted at 2026-08-09 22:53:29 on dist-test-slave-ncp9"
server_key: "29926b41f9ec70edcccb0e259c7f4505"
server_key_iv: "71749821fbafc4ef52cb9ab81b1a4df9"
server_key_version: "encryptionkey@0"
I20260809 22:53:29.504334 29299 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260809 22:53:29.507745 29317 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:53:29.509038 29299 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 22:53:29.509207 29299 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315935617919-15814-0/minicluster-data/ts-0/data,/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315935617919-15814-0/minicluster-data/ts-0/wal
uuid: "9820493cc168460eaeea066a77cccd4d"
format_stamp: "Formatted at 2026-08-09 22:53:29 on dist-test-slave-ncp9"
server_key: "29926b41f9ec70edcccb0e259c7f4505"
server_key_iv: "71749821fbafc4ef52cb9ab81b1a4df9"
server_key_version: "encryptionkey@0"
I20260809 22:53:29.509377 29299 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315935617919-15814-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315935617919-15814-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315935617919-15814-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:53:29.520839 29299 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:53:29.524521 29299 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:53:29.524760 29299 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:53:29.525637 29299 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:53:29.526867 29299 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:53:29.526947 29299 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:53:29.527037 29299 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:53:29.527079 29299 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:53:29.557412 29299 rpc_server.cc:307] RPC server started. Bound to: 127.15.113.129:39729
I20260809 22:53:29.557461 29486 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.113.129:39729 every 8 connection(s)
I20260809 22:53:29.558720 29299 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315935617919-15814-0/minicluster-data/ts-0/data/info.pb
I20260809 22:53:29.559718 15814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskItYPJR/build/asan/bin/kudu as pid 29299
I20260809 22:53:29.559918 15814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315935617919-15814-0/minicluster-data/ts-0/wal/instance
I20260809 22:53:29.560602 15814 external_mini_cluster.cc:1428] Setting key 03b8416bd3c65ac7e6e1240fb6556f2f
Aug 09 22:53:29 dist-test-slave-ncp9 krb5kdc[29136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316009, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.113.129@KRBTEST.COM for kudu/127.15.113.190@KRBTEST.COM
I20260809 22:53:29.578145 29490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:29.561312 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.113.129:46583 (local address 127.15.113.190:44011)
0809 22:53:29.562347 (+ 1035us) server_negotiation.cc:206] Beginning negotiation
0809 22:53:29.562353 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:53:29.562800 (+ 447us) server_negotiation.cc:407] Connection header received
0809 22:53:29.564739 (+ 1939us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:53:29.564745 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:53:29.564823 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:53:29.564969 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:53:29.566349 (+ 1380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:29.567282 (+ 933us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:29.568736 (+ 1454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:29.569296 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:29.569420 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:29.572523 (+ 3103us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:53:29.572529 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:53:29.572532 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:53:29.572568 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:53:29.575202 (+ 2634us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:53:29.575957 (+ 755us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:53:29.575960 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:53:29.575963 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:53:29.576072 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:53:29.576503 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:53:29.576506 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:53:29.576508 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:53:29.576752 (+ 244us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:53:29.576854 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0809 22:53:29.577717 (+ 863us) server_negotiation.cc:299] Negotiation successful
0809 22:53:29.577888 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":339,"thread_start_us":222,"threads_started":1}
I20260809 22:53:29.579501 29492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:29.561857 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:44011 (local address 127.15.113.129:46583)
0809 22:53:29.562641 (+ 784us) negotiation.cc:107] Waiting for socket to connect
0809 22:53:29.562688 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0809 22:53:29.564484 (+ 1796us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:53:29.565133 (+ 649us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:53:29.565140 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:53:29.565633 (+ 493us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:53:29.566185 (+ 552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:53:29.566203 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:53:29.567416 (+ 1213us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:53:29.567419 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:53:29.568595 (+ 1176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:53:29.568611 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:53:29.569481 (+ 870us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:53:29.569486 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:53:29.569682 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:29.570281 (+ 599us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:53:29.570302 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:53:29.572322 (+ 2020us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:53:29.575332 (+ 3010us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:53:29.575340 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:53:29.575352 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:53:29.575866 (+ 514us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:53:29.576175 (+ 309us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:53:29.576179 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:53:29.576182 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:53:29.576414 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:53:29.576882 (+ 468us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:53:29.576888 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:53:29.577332 (+ 444us) client_negotiation.cc:769] Sending connection context
0809 22:53:29.577644 (+ 312us) client_negotiation.cc:240] Negotiation successful
0809 22:53:29.578098 (+ 454us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":526,"thread_start_us":324,"threads_started":1}
I20260809 22:53:29.581002 29488 heartbeater.cc:347] Connected to a master server at 127.15.113.190:44011
I20260809 22:53:29.581296 29488 heartbeater.cc:464] Registering TS with master...
I20260809 22:53:29.581954 29488 heartbeater.cc:511] Master 127.15.113.190:44011 requested a full tablet report, sending...
I20260809 22:53:29.583956 29199 ts_manager.cc:194] Registered new tserver with Master: 9820493cc168460eaeea066a77cccd4d (127.15.113.129:39729)
I20260809 22:53:29.585460 29199 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.113.129@KRBTEST.COM'} at 127.15.113.129:46583
WARNING: no policy specified for kudu/127.15.113.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.113.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.113.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.130.keytab.
Entry for principal kudu/127.15.113.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.130.keytab.
I20260809 22:53:29.617780 15814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskItYPJR/build/asan/bin/kudu
/tmp/dist-test-taskItYPJR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315935617919-15814-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315935617919-15814-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315935617919-15814-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315935617919-15814-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.113.130:0
--local_ip_for_outbound_sockets=127.15.113.130
--webserver_interface=127.15.113.130
--webserver_port=0
--tserver_master_addrs=127.15.113.190:44011
--keytab_file=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.130.keytab
--principal=kudu/127.15.113.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.113.148:40757
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315935617919-15814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315935617919-15814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315935617919-15814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:53:29.942111 29495 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:53:29.942404 29495 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:53:29.942528 29495 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:53:29.949501 29495 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:53:29.949602 29495 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:53:29.949755 29495 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.113.130
I20260809 22:53:29.958989 29495 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.113.148:40757
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315935617919-15814-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315935617919-15814-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.130.keytab
--principal=kudu/127.15.113.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.113.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315935617919-15814-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.113.130
--webserver_port=0
--tserver_master_addrs=127.15.113.190:44011
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.113.130
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315935617919-15814-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:53:29.960155 29495 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:53:29.961422 29495 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:53:29.974119 29505 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:53:29.974180 29506 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:53:29.974421 29511 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:53:29.975884 29495 server_base.cc:1034] running on GCE node
I20260809 22:53:29.976516 29495 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:53:29.977479 29495 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:53:29.978732 29495 hybrid_clock.cc:648] HybridClock initialized: now 1786316009978677 us; error 46 us; skew 500 ppm
I20260809 22:53:29.979031 29495 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:53:29 dist-test-slave-ncp9 krb5kdc[29136](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316009, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.113.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:53:29.982071 29495 init.cc:377] Logged in from keytab as kudu/127.15.113.130@KRBTEST.COM (short username kudu)
I20260809 22:53:29.983498 29495 webserver.cc:460] Webserver started at http://127.15.113.130:39855/ using document root <none> and password file <none>
I20260809 22:53:29.984171 29495 fs_manager.cc:359] Metadata directory not provided
I20260809 22:53:29.984277 29495 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:53:29.984568 29495 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:53:29.987411 29495 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315935617919-15814-0/minicluster-data/ts-1/data/instance:
uuid: "1050e47141ab4bcfaa21f78e7ada114e"
format_stamp: "Formatted at 2026-08-09 22:53:29 on dist-test-slave-ncp9"
server_key: "7a7b8479326114e67e88088b91850d8f"
server_key_iv: "4ca3121b6e8119e1def7da42190637ad"
server_key_version: "encryptionkey@0"
I20260809 22:53:29.988216 29495 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315935617919-15814-0/minicluster-data/ts-1/wal/instance:
uuid: "1050e47141ab4bcfaa21f78e7ada114e"
format_stamp: "Formatted at 2026-08-09 22:53:29 on dist-test-slave-ncp9"
server_key: "7a7b8479326114e67e88088b91850d8f"
server_key_iv: "4ca3121b6e8119e1def7da42190637ad"
server_key_version: "encryptionkey@0"
I20260809 22:53:29.993229 29495 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.008s sys 0.000s
I20260809 22:53:29.996464 29520 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:53:29.997740 29495 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 22:53:29.997895 29495 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315935617919-15814-0/minicluster-data/ts-1/data,/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315935617919-15814-0/minicluster-data/ts-1/wal
uuid: "1050e47141ab4bcfaa21f78e7ada114e"
format_stamp: "Formatted at 2026-08-09 22:53:29 on dist-test-slave-ncp9"
server_key: "7a7b8479326114e67e88088b91850d8f"
server_key_iv: "4ca3121b6e8119e1def7da42190637ad"
server_key_version: "encryptionkey@0"
I20260809 22:53:29.998076 29495 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315935617919-15814-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315935617919-15814-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315935617919-15814-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:53:30.011205 29495 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:53:30.015100 29495 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:53:30.015326 29495 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:53:30.016150 29495 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:53:30.017490 29495 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:53:30.017562 29495 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:53:30.017678 29495 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:53:30.017726 29495 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:53:30.047876 29495 rpc_server.cc:307] RPC server started. Bound to: 127.15.113.130:37943
I20260809 22:53:30.047915 29697 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.113.130:37943 every 8 connection(s)
I20260809 22:53:30.049243 29495 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315935617919-15814-0/minicluster-data/ts-1/data/info.pb
I20260809 22:53:30.059019 15814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskItYPJR/build/asan/bin/kudu as pid 29495
I20260809 22:53:30.059211 15814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315935617919-15814-0/minicluster-data/ts-1/wal/instance
I20260809 22:53:30.059656 15814 external_mini_cluster.cc:1428] Setting key 5051ae53184b3ecc54a222a1bbaf27a5
Aug 09 22:53:30 dist-test-slave-ncp9 krb5kdc[29136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316009, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.113.130@KRBTEST.COM for kudu/127.15.113.190@KRBTEST.COM
I20260809 22:53:30.068554 29490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:30.051774 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.113.130:53287 (local address 127.15.113.190:44011)
0809 22:53:30.051946 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0809 22:53:30.051951 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:53:30.053156 (+ 1205us) server_negotiation.cc:407] Connection header received
0809 22:53:30.054971 (+ 1815us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:53:30.054976 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:53:30.055058 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:53:30.055158 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:53:30.056621 (+ 1463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:30.057400 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:30.059121 (+ 1721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:30.059578 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:30.059691 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:30.063186 (+ 3495us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:53:30.063191 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:53:30.063195 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:53:30.063225 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:53:30.065768 (+ 2543us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:53:30.066427 (+ 659us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:53:30.066433 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:53:30.066435 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:53:30.066555 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:53:30.066947 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:53:30.066952 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:53:30.066955 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:53:30.067227 (+ 272us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:53:30.067355 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0809 22:53:30.068003 (+ 648us) server_negotiation.cc:299] Negotiation successful
0809 22:53:30.068238 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260809 22:53:30.069430 29702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:30.052229 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:44011 (local address 127.15.113.130:53287)
0809 22:53:30.052995 (+ 766us) negotiation.cc:107] Waiting for socket to connect
0809 22:53:30.053043 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0809 22:53:30.054701 (+ 1658us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:53:30.055388 (+ 687us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:53:30.055396 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:53:30.055901 (+ 505us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:53:30.056448 (+ 547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:53:30.056465 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:53:30.057560 (+ 1095us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:53:30.057564 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:53:30.058952 (+ 1388us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:53:30.058968 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:53:30.059820 (+ 852us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:53:30.059828 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:53:30.060102 (+ 274us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:30.060780 (+ 678us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:53:30.060808 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:53:30.062981 (+ 2173us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:53:30.065894 (+ 2913us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:53:30.065903 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:53:30.065918 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:53:30.066316 (+ 398us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:53:30.066663 (+ 347us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:53:30.066668 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:53:30.066671 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:53:30.066843 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:53:30.067346 (+ 503us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:53:30.067353 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:53:30.067669 (+ 316us) client_negotiation.cc:769] Sending connection context
0809 22:53:30.067899 (+ 230us) client_negotiation.cc:240] Negotiation successful
0809 22:53:30.068372 (+ 473us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":491,"thread_start_us":298,"threads_started":1}
I20260809 22:53:30.070806 29699 heartbeater.cc:347] Connected to a master server at 127.15.113.190:44011
I20260809 22:53:30.071087 29699 heartbeater.cc:464] Registering TS with master...
I20260809 22:53:30.071764 29699 heartbeater.cc:511] Master 127.15.113.190:44011 requested a full tablet report, sending...
I20260809 22:53:30.073141 29199 ts_manager.cc:194] Registered new tserver with Master: 1050e47141ab4bcfaa21f78e7ada114e (127.15.113.130:37943)
I20260809 22:53:30.073927 29199 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.113.130@KRBTEST.COM'} at 127.15.113.130:53287
WARNING: no policy specified for kudu/127.15.113.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.113.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.113.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.131.keytab.
Entry for principal kudu/127.15.113.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.131.keytab.
I20260809 22:53:30.115479 15814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskItYPJR/build/asan/bin/kudu
/tmp/dist-test-taskItYPJR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315935617919-15814-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315935617919-15814-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315935617919-15814-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315935617919-15814-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.113.131:0
--local_ip_for_outbound_sockets=127.15.113.131
--webserver_interface=127.15.113.131
--webserver_port=0
--tserver_master_addrs=127.15.113.190:44011
--keytab_file=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.131.keytab
--principal=kudu/127.15.113.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.113.148:40757
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315935617919-15814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315935617919-15814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315935617919-15814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:53:30.434211 29709 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:53:30.434490 29709 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:53:30.434593 29709 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:53:30.441586 29709 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:53:30.441694 29709 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:53:30.441807 29709 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.113.131
I20260809 22:53:30.451009 29709 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.113.148:40757
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315935617919-15814-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315935617919-15814-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.131.keytab
--principal=kudu/127.15.113.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.113.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315935617919-15814-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.113.131
--webserver_port=0
--tserver_master_addrs=127.15.113.190:44011
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.113.131
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315935617919-15814-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:53:30.452092 29709 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:53:30.453331 29709 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:53:30.465591 29714 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:53:30.465685 29715 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:53:30.465680 29719 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:53:30.466959 29709 server_base.cc:1034] running on GCE node
I20260809 22:53:30.467499 29709 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:53:30.468359 29709 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:53:30.469551 29709 hybrid_clock.cc:648] HybridClock initialized: now 1786316010469509 us; error 27 us; skew 500 ppm
I20260809 22:53:30.469846 29709 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:53:30 dist-test-slave-ncp9 krb5kdc[29136](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316010, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.113.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:53:30.472728 29709 init.cc:377] Logged in from keytab as kudu/127.15.113.131@KRBTEST.COM (short username kudu)
I20260809 22:53:30.474145 29709 webserver.cc:460] Webserver started at http://127.15.113.131:38449/ using document root <none> and password file <none>
I20260809 22:53:30.474778 29709 fs_manager.cc:359] Metadata directory not provided
I20260809 22:53:30.474898 29709 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:53:30.475190 29709 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:53:30.478093 29709 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315935617919-15814-0/minicluster-data/ts-2/data/instance:
uuid: "b3daf1cf1f124f749b4f928727aa586a"
format_stamp: "Formatted at 2026-08-09 22:53:30 on dist-test-slave-ncp9"
server_key: "70816cd73a69ebf7035dfea3396261b2"
server_key_iv: "f6811e9ef5aa99cad35860677034d145"
server_key_version: "encryptionkey@0"
I20260809 22:53:30.478868 29709 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315935617919-15814-0/minicluster-data/ts-2/wal/instance:
uuid: "b3daf1cf1f124f749b4f928727aa586a"
format_stamp: "Formatted at 2026-08-09 22:53:30 on dist-test-slave-ncp9"
server_key: "70816cd73a69ebf7035dfea3396261b2"
server_key_iv: "f6811e9ef5aa99cad35860677034d145"
server_key_version: "encryptionkey@0"
I20260809 22:53:30.483781 29709 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260809 22:53:30.486989 29731 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:53:30.488159 29709 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260809 22:53:30.488312 29709 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315935617919-15814-0/minicluster-data/ts-2/data,/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315935617919-15814-0/minicluster-data/ts-2/wal
uuid: "b3daf1cf1f124f749b4f928727aa586a"
format_stamp: "Formatted at 2026-08-09 22:53:30 on dist-test-slave-ncp9"
server_key: "70816cd73a69ebf7035dfea3396261b2"
server_key_iv: "f6811e9ef5aa99cad35860677034d145"
server_key_version: "encryptionkey@0"
I20260809 22:53:30.488494 29709 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315935617919-15814-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315935617919-15814-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315935617919-15814-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:53:30.515579 29709 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:53:30.519457 29709 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:53:30.519716 29709 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:53:30.520627 29709 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:53:30.521932 29709 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:53:30.522011 29709 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:53:30.522135 29709 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:53:30.522200 29709 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:53:30.551895 29709 rpc_server.cc:307] RPC server started. Bound to: 127.15.113.131:38311
I20260809 22:53:30.551939 29897 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.113.131:38311 every 8 connection(s)
I20260809 22:53:30.554082 29709 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315935617919-15814-0/minicluster-data/ts-2/data/info.pb
I20260809 22:53:30.557327 15814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskItYPJR/build/asan/bin/kudu as pid 29709
I20260809 22:53:30.557494 15814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315935617919-15814-0/minicluster-data/ts-2/wal/instance
I20260809 22:53:30.558158 15814 external_mini_cluster.cc:1428] Setting key 5aab46fd1043c1dd2977d48913484b98
Aug 09 22:53:30 dist-test-slave-ncp9 krb5kdc[29136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316010, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.113.131@KRBTEST.COM for kudu/127.15.113.190@KRBTEST.COM
I20260809 22:53:30.574764 29490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:30.556604 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.113.131:45055 (local address 127.15.113.190:44011)
0809 22:53:30.556813 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0809 22:53:30.556819 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:53:30.559576 (+ 2757us) server_negotiation.cc:407] Connection header received
0809 22:53:30.561887 (+ 2311us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:53:30.561892 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:53:30.561974 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:53:30.562059 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:53:30.563417 (+ 1358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:30.564153 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:30.565722 (+ 1569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:30.566123 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:30.566209 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:30.569282 (+ 3073us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:53:30.569287 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:53:30.569290 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:53:30.569321 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:53:30.571752 (+ 2431us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:53:30.572552 (+ 800us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:53:30.572556 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:53:30.572558 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:53:30.572700 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:53:30.573210 (+ 510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:53:30.573215 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:53:30.573217 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:53:30.573574 (+ 357us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:53:30.573712 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0809 22:53:30.574344 (+ 632us) server_negotiation.cc:299] Negotiation successful
0809 22:53:30.574510 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260809 22:53:30.575590 29902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:30.557248 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:44011 (local address 127.15.113.131:45055)
0809 22:53:30.559417 (+ 2169us) negotiation.cc:107] Waiting for socket to connect
0809 22:53:30.559471 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0809 22:53:30.561642 (+ 2171us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:53:30.562261 (+ 619us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:53:30.562268 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:53:30.562756 (+ 488us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:53:30.563289 (+ 533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:53:30.563307 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:53:30.564264 (+ 957us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:53:30.564268 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:53:30.565582 (+ 1314us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:53:30.565595 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:53:30.566292 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:53:30.566297 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:53:30.566478 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:30.567047 (+ 569us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:53:30.567067 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:53:30.569084 (+ 2017us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:53:30.571875 (+ 2791us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:53:30.571884 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:53:30.571899 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:53:30.572422 (+ 523us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:53:30.572810 (+ 388us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:53:30.572813 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:53:30.572816 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:53:30.573084 (+ 268us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:53:30.573712 (+ 628us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:53:30.573718 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:53:30.574001 (+ 283us) client_negotiation.cc:769] Sending connection context
0809 22:53:30.574221 (+ 220us) client_negotiation.cc:240] Negotiation successful
0809 22:53:30.574578 (+ 357us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1886,"thread_start_us":1685,"threads_started":1}
I20260809 22:53:30.576690 29898 heartbeater.cc:347] Connected to a master server at 127.15.113.190:44011
I20260809 22:53:30.576998 29898 heartbeater.cc:464] Registering TS with master...
I20260809 22:53:30.577673 29898 heartbeater.cc:511] Master 127.15.113.190:44011 requested a full tablet report, sending...
I20260809 22:53:30.579010 29199 ts_manager.cc:194] Registered new tserver with Master: b3daf1cf1f124f749b4f928727aa586a (127.15.113.131:38311)
I20260809 22:53:30.579751 29199 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.113.131@KRBTEST.COM'} at 127.15.113.131:45055
I20260809 22:53:30.585407 15814 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 22:53:30.588203 29488 heartbeater.cc:503] Master 127.15.113.190:44011 was elected leader, sending a full tablet report...
WARNING: no policy specified for oryx/127.15.113.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.15.113.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.15.113.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315935617919-15814-0/krb5kdc/oryx_127.15.113.129.keytab.
Entry for principal oryx/127.15.113.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315935617919-15814-0/krb5kdc/oryx_127.15.113.129.keytab.
WARNING: no policy specified for oryx/127.15.113.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.15.113.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.15.113.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315935617919-15814-0/krb5kdc/oryx_127.15.113.130.keytab.
Entry for principal oryx/127.15.113.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315935617919-15814-0/krb5kdc/oryx_127.15.113.130.keytab.
WARNING: no policy specified for oryx/127.15.113.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.15.113.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.15.113.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315935617919-15814-0/krb5kdc/oryx_127.15.113.131.keytab.
Entry for principal oryx/127.15.113.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315935617919-15814-0/krb5kdc/oryx_127.15.113.131.keytab.
I20260809 22:53:30.734972 15814 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskItYPJR/build/asan/bin/kudu pid 29299
Aug 09 22:53:30 dist-test-slave-ncp9 krb5kdc[29136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316008, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.113.129@KRBTEST.COM
I20260809 22:53:30.750592 29915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:30.736102 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44026 (local address 127.15.113.129:39729)
0809 22:53:30.736813 (+ 711us) server_negotiation.cc:206] Beginning negotiation
0809 22:53:30.736820 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:53:30.736846 (+ 26us) server_negotiation.cc:407] Connection header received
0809 22:53:30.736939 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:53:30.736949 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:53:30.737168 (+ 219us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:53:30.737317 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:53:30.738636 (+ 1319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:30.739656 (+ 1020us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:30.740926 (+ 1270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:30.741501 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:30.741616 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:30.745475 (+ 3859us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:53:30.745486 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:53:30.745500 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:53:30.745536 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:53:30.747990 (+ 2454us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:53:30.748615 (+ 625us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:53:30.748622 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:53:30.748624 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:53:30.748760 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:53:30.749124 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:53:30.749128 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:53:30.749130 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:53:30.749461 (+ 331us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:53:30.749758 (+ 297us) server_negotiation.cc:1035] Waiting for connection context
0809 22:53:30.749998 (+ 240us) server_negotiation.cc:299] Negotiation successful
0809 22:53:30.750250 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":507,"thread_start_us":336,"threads_started":1}
I20260809 22:53:30.751847 29481 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:53:30.833318 15814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskItYPJR/build/asan/bin/kudu with pid 29299
I20260809 22:53:30.860783 15814 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskItYPJR/build/asan/bin/kudu pid 29495
Aug 09 22:53:30 dist-test-slave-ncp9 krb5kdc[29136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316008, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.113.130@KRBTEST.COM
I20260809 22:53:30.876521 29918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:30.861745 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39178 (local address 127.15.113.130:37943)
0809 22:53:30.862327 (+ 582us) server_negotiation.cc:206] Beginning negotiation
0809 22:53:30.862334 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:53:30.862361 (+ 27us) server_negotiation.cc:407] Connection header received
0809 22:53:30.862432 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:53:30.862439 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:53:30.862640 (+ 201us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:53:30.862778 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:53:30.863959 (+ 1181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:30.865000 (+ 1041us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:30.866372 (+ 1372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:30.866987 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:30.867115 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:30.871731 (+ 4616us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:53:30.871743 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:53:30.871758 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:53:30.871790 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:53:30.874323 (+ 2533us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:53:30.874917 (+ 594us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:53:30.874923 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:53:30.874925 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:53:30.875049 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:53:30.875312 (+ 263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:53:30.875315 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:53:30.875317 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:53:30.875629 (+ 312us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:53:30.875859 (+ 230us) server_negotiation.cc:1035] Waiting for connection context
0809 22:53:30.876052 (+ 193us) server_negotiation.cc:299] Negotiation successful
0809 22:53:30.876238 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":418,"thread_start_us":277,"threads_started":1}
I20260809 22:53:30.877846 29687 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:53:30.957785 15814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskItYPJR/build/asan/bin/kudu with pid 29495
I20260809 22:53:30.980856 15814 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskItYPJR/build/asan/bin/kudu pid 29709
Aug 09 22:53:30 dist-test-slave-ncp9 krb5kdc[29136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316008, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.113.131@KRBTEST.COM
I20260809 22:53:30.996870 29920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:30.981936 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47656 (local address 127.15.113.131:38311)
0809 22:53:30.982759 (+ 823us) server_negotiation.cc:206] Beginning negotiation
0809 22:53:30.982766 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:53:30.982798 (+ 32us) server_negotiation.cc:407] Connection header received
0809 22:53:30.982863 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:53:30.982870 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:53:30.983094 (+ 224us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:53:30.983227 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:53:30.984499 (+ 1272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:30.985649 (+ 1150us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:30.986884 (+ 1235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:30.987412 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:30.987508 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:30.991670 (+ 4162us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:53:30.991681 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:53:30.991696 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:53:30.991727 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:53:30.994578 (+ 2851us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:53:30.995167 (+ 589us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:53:30.995173 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:53:30.995175 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:53:30.995303 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:53:30.995580 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:53:30.995583 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:53:30.995585 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:53:30.995905 (+ 320us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:53:30.996118 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0809 22:53:30.996333 (+ 215us) server_negotiation.cc:299] Negotiation successful
0809 22:53:30.996520 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":675,"thread_start_us":483,"threads_started":1}
I20260809 22:53:30.998150 29893 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:53:31.078939 15814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskItYPJR/build/asan/bin/kudu with pid 29709
I20260809 22:53:31.099928 15814 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskItYPJR/build/asan/bin/kudu pid 29152
I20260809 22:53:31.101145 29246 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:53:31.203068 15814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskItYPJR/build/asan/bin/kudu with pid 29152
I20260809 22:53:31.230527 15814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskItYPJR/build/asan/bin/kudu
/tmp/dist-test-taskItYPJR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315935617919-15814-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315935617919-15814-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315935617919-15814-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315935617919-15814-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.113.190:44011
--webserver_interface=127.15.113.190
--webserver_port=34253
--keytab_file=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.190.keytab
--principal=kudu/127.15.113.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.113.148:40757
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.113.190:44011
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315935617919-15814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315935617919-15814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315935617919-15814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:53:31.566387 29924 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:53:31.566670 29924 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:53:31.566776 29924 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:53:31.573956 29924 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 22:53:31.574059 29924 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:53:31.574111 29924 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:53:31.574167 29924 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 22:53:31.574209 29924 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 22:53:31.583695 29924 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.113.148:40757
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315935617919-15814-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315935617919-15814-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.113.190:44011
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.190.keytab
--principal=kudu/127.15.113.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.113.190:44011
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315935617919-15814-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.113.190
--webserver_port=34253
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315935617919-15814-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:53:31.584949 29924 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:53:31.586170 29924 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:53:31.597793 29931 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:53:31.597710 29934 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:53:31.597749 29932 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:53:31.598579 29924 server_base.cc:1034] running on GCE node
I20260809 22:53:31.599323 29924 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:53:31.600641 29924 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:53:31.601861 29924 hybrid_clock.cc:648] HybridClock initialized: now 1786316011601797 us; error 49 us; skew 500 ppm
I20260809 22:53:31.602227 29924 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:53:31 dist-test-slave-ncp9 krb5kdc[29136](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316011, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.113.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:53:31.605309 29924 init.cc:377] Logged in from keytab as kudu/127.15.113.190@KRBTEST.COM (short username kudu)
I20260809 22:53:31.606693 29924 webserver.cc:460] Webserver started at http://127.15.113.190:34253/ using document root <none> and password file <none>
I20260809 22:53:31.607355 29924 fs_manager.cc:359] Metadata directory not provided
I20260809 22:53:31.607486 29924 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:53:31.612777 29924 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.001s
I20260809 22:53:31.615525 29943 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:53:31.616847 29924 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 22:53:31.617076 29924 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315935617919-15814-0/minicluster-data/master-0/data,/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315935617919-15814-0/minicluster-data/master-0/wal
uuid: "2ea0d8ef17c6474690168da53ac2015c"
format_stamp: "Formatted at 2026-08-09 22:53:28 on dist-test-slave-ncp9"
server_key: "4c979b2d913f16758cf23e4ee1af2901"
server_key_iv: "ed80cae4303660da1b26ae342fa5ed26"
server_key_version: "encryptionkey@0"
I20260809 22:53:31.618041 29924 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315935617919-15814-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315935617919-15814-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315935617919-15814-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:53:31.631871 29924 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:53:31.635671 29924 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:53:31.636042 29924 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:53:31.653311 29924 rpc_server.cc:307] RPC server started. Bound to: 127.15.113.190:44011
I20260809 22:53:31.653347 30015 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.113.190:44011 every 8 connection(s)
I20260809 22:53:31.654742 29924 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315935617919-15814-0/minicluster-data/master-0/data/info.pb
I20260809 22:53:31.655212 15814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskItYPJR/build/asan/bin/kudu as pid 29924
I20260809 22:53:31.656111 15814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskItYPJR/build/asan/bin/kudu
/tmp/dist-test-taskItYPJR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315935617919-15814-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315935617919-15814-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315935617919-15814-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315935617919-15814-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.113.129:39729
--local_ip_for_outbound_sockets=127.15.113.129
--tserver_master_addrs=127.15.113.190:44011
--webserver_port=35445
--webserver_interface=127.15.113.129
--keytab_file=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.129.keytab
--principal=kudu/127.15.113.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.113.148:40757
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.15.113.129
--keytab_file=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315935617919-15814-0/krb5kdc/oryx_127.15.113.129.keytab with env {KRB5CCNAME=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315935617919-15814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315935617919-15814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315935617919-15814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 22:53:31.661650 30016 sys_catalog.cc:263] Verifying existing consensus state
I20260809 22:53:31.665254 30016 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2ea0d8ef17c6474690168da53ac2015c: Bootstrap starting.
I20260809 22:53:31.684008 30016 log.cc:824] T 00000000000000000000000000000000 P 2ea0d8ef17c6474690168da53ac2015c: Log is configured to *not* fsync() on all Append() calls
I20260809 22:53:31.692762 30016 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2ea0d8ef17c6474690168da53ac2015c: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 22:53:31.693279 30016 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2ea0d8ef17c6474690168da53ac2015c: Bootstrap complete.
I20260809 22:53:31.702010 30016 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2ea0d8ef17c6474690168da53ac2015c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ea0d8ef17c6474690168da53ac2015c" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 44011 } }
I20260809 22:53:31.702822 30016 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2ea0d8ef17c6474690168da53ac2015c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ea0d8ef17c6474690168da53ac2015c, State: Initialized, Role: FOLLOWER
I20260809 22:53:31.703414 30016 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2ea0d8ef17c6474690168da53ac2015c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ea0d8ef17c6474690168da53ac2015c" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 44011 } }
I20260809 22:53:31.703639 30016 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2ea0d8ef17c6474690168da53ac2015c [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 22:53:31.703754 30016 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2ea0d8ef17c6474690168da53ac2015c [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 22:53:31.703913 30016 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2ea0d8ef17c6474690168da53ac2015c [term 1 FOLLOWER]: Advancing to term 2
I20260809 22:53:31.707579 30016 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2ea0d8ef17c6474690168da53ac2015c [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ea0d8ef17c6474690168da53ac2015c" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 44011 } }
I20260809 22:53:31.708066 30016 leader_election.cc:302] T 00000000000000000000000000000000 P 2ea0d8ef17c6474690168da53ac2015c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2ea0d8ef17c6474690168da53ac2015c; no voters:
I20260809 22:53:31.708666 30016 leader_election.cc:288] T 00000000000000000000000000000000 P 2ea0d8ef17c6474690168da53ac2015c [CANDIDATE]: Term 2 election: Requested vote from peers
I20260809 22:53:31.708863 30024 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2ea0d8ef17c6474690168da53ac2015c [term 2 FOLLOWER]: Leader election won for term 2
I20260809 22:53:31.709823 30024 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2ea0d8ef17c6474690168da53ac2015c [term 2 LEADER]: Becoming Leader. State: Replica: 2ea0d8ef17c6474690168da53ac2015c, State: Running, Role: LEADER
I20260809 22:53:31.710638 30016 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2ea0d8ef17c6474690168da53ac2015c [sys.catalog]: configured and running, proceeding with master startup.
I20260809 22:53:31.710552 30024 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2ea0d8ef17c6474690168da53ac2015c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ea0d8ef17c6474690168da53ac2015c" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 44011 } }
I20260809 22:53:31.713636 30030 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2ea0d8ef17c6474690168da53ac2015c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2ea0d8ef17c6474690168da53ac2015c. Latest consensus state: current_term: 2 leader_uuid: "2ea0d8ef17c6474690168da53ac2015c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ea0d8ef17c6474690168da53ac2015c" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 44011 } } }
I20260809 22:53:31.714022 30030 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2ea0d8ef17c6474690168da53ac2015c [sys.catalog]: This master's current role is: LEADER
I20260809 22:53:31.714473 30029 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2ea0d8ef17c6474690168da53ac2015c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "2ea0d8ef17c6474690168da53ac2015c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ea0d8ef17c6474690168da53ac2015c" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 44011 } } }
I20260809 22:53:31.714766 30029 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2ea0d8ef17c6474690168da53ac2015c [sys.catalog]: This master's current role is: LEADER
I20260809 22:53:31.721505 30042 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 22:53:31.727387 30042 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 22:53:31.728789 30042 catalog_manager.cc:1259] Loaded cluster ID: e087c41c879541c99dcb01859414f64e
I20260809 22:53:31.728950 30042 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 22:53:31.732826 30042 catalog_manager.cc:1504] Loading token signing keys...
I20260809 22:53:31.734373 30042 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 2ea0d8ef17c6474690168da53ac2015c: Loaded TSK: 0
I20260809 22:53:31.736057 30042 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260809 22:53:31.999866 30020 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:53:32.000161 30020 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:53:32.000296 30020 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:53:32.007648 30020 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:53:32.007778 30020 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:53:32.007929 30020 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.113.129
I20260809 22:53:32.017751 30020 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.113.148:40757
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315935617919-15814-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315935617919-15814-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315935617919-15814-0/krb5kdc/oryx_127.15.113.129.keytab
--principal=oryx/127.15.113.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.113.129:39729
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315935617919-15814-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.113.129
--webserver_port=35445
--tserver_master_addrs=127.15.113.190:44011
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.113.129
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315935617919-15814-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:53:32.019030 30020 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:53:32.020300 30020 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:53:32.033396 30065 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:53:32.033607 30067 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:53:32.033842 30062 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:53:32.034161 30020 server_base.cc:1034] running on GCE node
I20260809 22:53:32.034718 30020 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:53:32.035650 30020 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:53:32.036857 30020 hybrid_clock.cc:648] HybridClock initialized: now 1786316012036814 us; error 38 us; skew 500 ppm
I20260809 22:53:32.037174 30020 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:53:32 dist-test-slave-ncp9 krb5kdc[29136](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316012, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.113.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:53:32.040167 30020 init.cc:377] Logged in from keytab as oryx/127.15.113.129@KRBTEST.COM (short username oryx)
I20260809 22:53:32.041684 30020 webserver.cc:460] Webserver started at http://127.15.113.129:35445/ using document root <none> and password file <none>
I20260809 22:53:32.042378 30020 fs_manager.cc:359] Metadata directory not provided
I20260809 22:53:32.042498 30020 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:53:32.047745 30020 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260809 22:53:32.050303 30076 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:53:32.051571 30020 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 22:53:32.051715 30020 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315935617919-15814-0/minicluster-data/ts-0/data,/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315935617919-15814-0/minicluster-data/ts-0/wal
uuid: "9820493cc168460eaeea066a77cccd4d"
format_stamp: "Formatted at 2026-08-09 22:53:29 on dist-test-slave-ncp9"
server_key: "29926b41f9ec70edcccb0e259c7f4505"
server_key_iv: "71749821fbafc4ef52cb9ab81b1a4df9"
server_key_version: "encryptionkey@0"
I20260809 22:53:32.052654 30020 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315935617919-15814-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315935617919-15814-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315935617919-15814-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:53:32.074103 30020 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:53:32.078042 30020 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:53:32.078279 30020 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:53:32.079190 30020 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:53:32.080466 30020 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:53:32.080567 30020 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:53:32.080674 30020 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:53:32.080751 30020 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:53:32.111021 30020 rpc_server.cc:307] RPC server started. Bound to: 127.15.113.129:39729
I20260809 22:53:32.111104 30249 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.113.129:39729 every 8 connection(s)
I20260809 22:53:32.112380 30020 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315935617919-15814-0/minicluster-data/ts-0/data/info.pb
I20260809 22:53:32.113458 15814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskItYPJR/build/asan/bin/kudu as pid 30020
I20260809 22:53:32.114207 15814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskItYPJR/build/asan/bin/kudu
/tmp/dist-test-taskItYPJR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315935617919-15814-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315935617919-15814-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315935617919-15814-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315935617919-15814-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.113.130:37943
--local_ip_for_outbound_sockets=127.15.113.130
--tserver_master_addrs=127.15.113.190:44011
--webserver_port=39855
--webserver_interface=127.15.113.130
--keytab_file=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.130.keytab
--principal=kudu/127.15.113.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.113.148:40757
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.15.113.130
--keytab_file=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315935617919-15814-0/krb5kdc/oryx_127.15.113.130.keytab with env {KRB5CCNAME=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315935617919-15814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315935617919-15814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315935617919-15814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 22:53:32 dist-test-slave-ncp9 krb5kdc[29136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.113.129@KRBTEST.COM for oryx/127.15.113.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 22:53:32 dist-test-slave-ncp9 krb5kdc[29136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.113.129@KRBTEST.COM for oryx/127.15.113.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 22:53:32.130632 30253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:32.115692 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:44011 (local address 127.15.113.129:47467)
0809 22:53:32.116480 (+ 788us) negotiation.cc:107] Waiting for socket to connect
0809 22:53:32.116527 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0809 22:53:32.118406 (+ 1879us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:53:32.119751 (+ 1345us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:53:32.119762 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:53:32.120352 (+ 590us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:53:32.120877 (+ 525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:53:32.120934 (+ 57us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:53:32.122704 (+ 1770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:53:32.122710 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:53:32.123940 (+ 1230us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:53:32.123953 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:53:32.124989 (+ 1036us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:53:32.124999 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:53:32.125272 (+ 273us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:32.125868 (+ 596us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:53:32.125894 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:53:32.129432 (+ 3538us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.113.190:44011: Server oryx/127.15.113.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":508,"thread_start_us":315,"threads_started":1}
W20260809 22:53:32.130887 30255 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:53:32.115807 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.113.129:47467 (local address 127.15.113.190:44011)
0809 22:53:32.117339 (+ 1532us) server_negotiation.cc:206] Beginning negotiation
0809 22:53:32.117354 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0809 22:53:32.117406 (+ 52us) server_negotiation.cc:407] Connection header received
0809 22:53:32.118934 (+ 1528us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:53:32.118947 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:53:32.119260 (+ 313us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:53:32.119601 (+ 341us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:53:32.121116 (+ 1515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:32.122548 (+ 1432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:32.124076 (+ 1528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:32.124717 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:32.124918 (+ 201us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:32.129802 (+ 4884us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.113.129:47467: BlockingRecv error: recv got EOF from 127.15.113.129:47467 (error 108)
Metrics: {"server-negotiator.queue_time_us":1293,"thread_start_us":1094,"threads_started":1}
W20260809 22:53:32.131419 30251 heartbeater.cc:644] Failed to heartbeat to 127.15.113.190:44011 (0 consecutive failures): Not authorized: Failed to ping master at 127.15.113.190:44011: Client connection negotiation failed: client connection to 127.15.113.190:44011: Server oryx/127.15.113.190@KRBTEST.COM not found in Kerberos database
Aug 09 22:53:32 dist-test-slave-ncp9 krb5kdc[29136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.113.129@KRBTEST.COM for oryx/127.15.113.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 22:53:32 dist-test-slave-ncp9 krb5kdc[29136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.113.129@KRBTEST.COM for oryx/127.15.113.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 22:53:32.139295 30253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:32.131967 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:44011 (local address 127.15.113.129:48151)
0809 22:53:32.132134 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0809 22:53:32.132152 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 22:53:32.132239 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:53:32.132589 (+ 350us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:53:32.132594 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:53:32.132943 (+ 349us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:53:32.133088 (+ 145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:53:32.133097 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:53:32.134054 (+ 957us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:53:32.134059 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:53:32.134924 (+ 865us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:53:32.134935 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:53:32.135596 (+ 661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:53:32.135602 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:53:32.135758 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:32.136264 (+ 506us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:53:32.136281 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:53:32.138995 (+ 2714us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.113.190:44011: Server oryx/127.15.113.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":41}
W20260809 22:53:32.139494 30255 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:53:32.132171 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.113.129:48151 (local address 127.15.113.190:44011)
0809 22:53:32.132340 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0809 22:53:32.132345 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:53:32.132358 (+ 13us) server_negotiation.cc:407] Connection header received
0809 22:53:32.132394 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:53:32.132398 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:53:32.132448 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:53:32.132524 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:53:32.133215 (+ 691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:32.133916 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:32.135062 (+ 1146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:32.135439 (+ 377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:32.135542 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:32.139201 (+ 3659us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.113.129:48151: BlockingRecv error: recv got EOF from 127.15.113.129:48151 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
Aug 09 22:53:32 dist-test-slave-ncp9 krb5kdc[29136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.113.129@KRBTEST.COM for oryx/127.15.113.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 22:53:32 dist-test-slave-ncp9 krb5kdc[29136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.113.129@KRBTEST.COM for oryx/127.15.113.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 22:53:32.147070 30253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:32.140060 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:44011 (local address 127.15.113.129:40217)
0809 22:53:32.140195 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0809 22:53:32.140213 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 22:53:32.140289 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:53:32.140657 (+ 368us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:53:32.140662 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:53:32.141001 (+ 339us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:53:32.141141 (+ 140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:53:32.141149 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:53:32.142070 (+ 921us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:53:32.142076 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:53:32.142970 (+ 894us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:53:32.142981 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:53:32.143635 (+ 654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:53:32.143641 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:53:32.143808 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:32.144290 (+ 482us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:53:32.144306 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:53:32.146809 (+ 2503us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.113.190:44011: Server oryx/127.15.113.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":40}
W20260809 22:53:32.147241 30255 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:53:32.140254 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.113.129:40217 (local address 127.15.113.190:44011)
0809 22:53:32.140418 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0809 22:53:32.140422 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 22:53:32.140435 (+ 13us) server_negotiation.cc:407] Connection header received
0809 22:53:32.140469 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:53:32.140472 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:53:32.140537 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:53:32.140613 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:53:32.141267 (+ 654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:32.141941 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:32.143101 (+ 1160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:32.143506 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:32.143611 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:32.147006 (+ 3395us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.113.129:40217: BlockingRecv error: recv got EOF from 127.15.113.129:40217 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
W20260809 22:53:32.147456 30251 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260809 22:53:32.442162 30256 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:53:32.442440 30256 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:53:32.442553 30256 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:53:32.449522 30256 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:53:32.449620 30256 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:53:32.449770 30256 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.113.130
I20260809 22:53:32.459363 30256 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.113.148:40757
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315935617919-15814-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315935617919-15814-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315935617919-15814-0/krb5kdc/oryx_127.15.113.130.keytab
--principal=oryx/127.15.113.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.113.130:37943
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315935617919-15814-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.113.130
--webserver_port=39855
--tserver_master_addrs=127.15.113.190:44011
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.113.130
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315935617919-15814-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:53:32.460587 30256 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:53:32.461867 30256 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:53:32.474464 30262 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:53:32.474509 30263 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:53:32.474519 30265 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:53:32.475340 30256 server_base.cc:1034] running on GCE node
I20260809 22:53:32.475929 30256 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:53:32.476799 30256 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:53:32.478025 30256 hybrid_clock.cc:648] HybridClock initialized: now 1786316012477973 us; error 42 us; skew 500 ppm
I20260809 22:53:32.478330 30256 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:53:32 dist-test-slave-ncp9 krb5kdc[29136](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316012, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.113.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:53:32.481383 30256 init.cc:377] Logged in from keytab as oryx/127.15.113.130@KRBTEST.COM (short username oryx)
I20260809 22:53:32.482720 30256 webserver.cc:460] Webserver started at http://127.15.113.130:39855/ using document root <none> and password file <none>
I20260809 22:53:32.483345 30256 fs_manager.cc:359] Metadata directory not provided
I20260809 22:53:32.483451 30256 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:53:32.488544 30256 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.001s
I20260809 22:53:32.491142 30275 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:53:32.492478 30256 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 22:53:32.492631 30256 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315935617919-15814-0/minicluster-data/ts-1/data,/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315935617919-15814-0/minicluster-data/ts-1/wal
uuid: "1050e47141ab4bcfaa21f78e7ada114e"
format_stamp: "Formatted at 2026-08-09 22:53:29 on dist-test-slave-ncp9"
server_key: "7a7b8479326114e67e88088b91850d8f"
server_key_iv: "4ca3121b6e8119e1def7da42190637ad"
server_key_version: "encryptionkey@0"
I20260809 22:53:32.493647 30256 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315935617919-15814-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315935617919-15814-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315935617919-15814-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:53:32.508733 30256 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:53:32.512603 30256 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:53:32.512848 30256 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:53:32.513839 30256 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:53:32.515127 30256 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:53:32.515232 30256 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:53:32.515338 30256 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:53:32.515406 30256 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:53:32.545292 30256 rpc_server.cc:307] RPC server started. Bound to: 127.15.113.130:37943
I20260809 22:53:32.545377 30440 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.113.130:37943 every 8 connection(s)
I20260809 22:53:32.546695 30256 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315935617919-15814-0/minicluster-data/ts-1/data/info.pb
I20260809 22:53:32.551867 15814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskItYPJR/build/asan/bin/kudu as pid 30256
I20260809 22:53:32.552577 15814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskItYPJR/build/asan/bin/kudu
/tmp/dist-test-taskItYPJR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315935617919-15814-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315935617919-15814-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315935617919-15814-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315935617919-15814-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.113.131:38311
--local_ip_for_outbound_sockets=127.15.113.131
--tserver_master_addrs=127.15.113.190:44011
--webserver_port=38449
--webserver_interface=127.15.113.131
--keytab_file=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.131.keytab
--principal=kudu/127.15.113.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.113.148:40757
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.15.113.131
--keytab_file=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315935617919-15814-0/krb5kdc/oryx_127.15.113.131.keytab with env {KRB5CCNAME=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315935617919-15814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315935617919-15814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315935617919-15814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 22:53:32 dist-test-slave-ncp9 krb5kdc[29136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.113.130@KRBTEST.COM for oryx/127.15.113.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 22:53:32 dist-test-slave-ncp9 krb5kdc[29136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.113.130@KRBTEST.COM for oryx/127.15.113.190@KRBTEST.COM, Server not found in Kerberos database
W20260809 22:53:32.563449 30255 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:53:32.549130 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.113.130:36013 (local address 127.15.113.190:44011)
0809 22:53:32.549330 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0809 22:53:32.549336 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:53:32.550565 (+ 1229us) server_negotiation.cc:407] Connection header received
0809 22:53:32.552695 (+ 2130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:53:32.552700 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:53:32.552790 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:53:32.552922 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:53:32.554846 (+ 1924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:32.555663 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:32.557739 (+ 2076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:32.558167 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:32.558261 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:32.563205 (+ 4944us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.113.130:36013: BlockingRecv error: recv got EOF from 127.15.113.130:36013 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
I20260809 22:53:32.564096 30444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:32.549706 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:44011 (local address 127.15.113.130:36013)
0809 22:53:32.550407 (+ 701us) negotiation.cc:107] Waiting for socket to connect
0809 22:53:32.550453 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0809 22:53:32.552393 (+ 1940us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:53:32.553174 (+ 781us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:53:32.553184 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:53:32.553866 (+ 682us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:53:32.554655 (+ 789us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:53:32.554681 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:53:32.555815 (+ 1134us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:53:32.555819 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:53:32.557609 (+ 1790us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:53:32.557623 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:53:32.558379 (+ 756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:53:32.558385 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:53:32.558634 (+ 249us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:32.559299 (+ 665us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:53:32.559326 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:53:32.562999 (+ 3673us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.113.190:44011: Server oryx/127.15.113.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":457,"thread_start_us":259,"threads_started":1}
W20260809 22:53:32.564669 30441 heartbeater.cc:644] Failed to heartbeat to 127.15.113.190:44011 (0 consecutive failures): Not authorized: Failed to ping master at 127.15.113.190:44011: Client connection negotiation failed: client connection to 127.15.113.190:44011: Server oryx/127.15.113.190@KRBTEST.COM not found in Kerberos database
Aug 09 22:53:32 dist-test-slave-ncp9 krb5kdc[29136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.113.130@KRBTEST.COM for oryx/127.15.113.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 22:53:32 dist-test-slave-ncp9 krb5kdc[29136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.113.130@KRBTEST.COM for oryx/127.15.113.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 22:53:32.572496 30444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:32.565273 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:44011 (local address 127.15.113.130:35829)
0809 22:53:32.565442 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0809 22:53:32.565460 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 22:53:32.565552 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:53:32.565843 (+ 291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:53:32.565847 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:53:32.566152 (+ 305us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:53:32.566284 (+ 132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:53:32.566292 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:53:32.567229 (+ 937us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:53:32.567233 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:53:32.568091 (+ 858us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:53:32.568102 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:53:32.568767 (+ 665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:53:32.568773 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:53:32.568957 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:32.569460 (+ 503us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:53:32.569477 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:53:32.572243 (+ 2766us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.113.190:44011: Server oryx/127.15.113.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":47}
W20260809 22:53:32.572594 30255 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:53:32.565352 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.113.130:35829 (local address 127.15.113.190:44011)
0809 22:53:32.565513 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0809 22:53:32.565518 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:53:32.565535 (+ 17us) server_negotiation.cc:407] Connection header received
0809 22:53:32.565664 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:53:32.565670 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:53:32.565729 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:53:32.565805 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:53:32.566416 (+ 611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:32.567097 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:32.568232 (+ 1135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:32.568634 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:32.568738 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:32.572418 (+ 3680us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.113.130:35829: BlockingRecv error: recv got EOF from 127.15.113.130:35829 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Aug 09 22:53:32 dist-test-slave-ncp9 krb5kdc[29136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.113.130@KRBTEST.COM for oryx/127.15.113.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 22:53:32 dist-test-slave-ncp9 krb5kdc[29136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.113.130@KRBTEST.COM for oryx/127.15.113.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 22:53:32.580116 30444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:32.573322 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:44011 (local address 127.15.113.130:34819)
0809 22:53:32.573470 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0809 22:53:32.573488 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 22:53:32.573579 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:53:32.573859 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:53:32.573863 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:53:32.574146 (+ 283us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:53:32.574280 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:53:32.574288 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:53:32.575203 (+ 915us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:53:32.575207 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:53:32.576069 (+ 862us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:53:32.576079 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:53:32.576723 (+ 644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:53:32.576729 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:53:32.576938 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:32.577417 (+ 479us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:53:32.577434 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:53:32.579844 (+ 2410us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.113.190:44011: Server oryx/127.15.113.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":47}
W20260809 22:53:32.580219 30255 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:53:32.573412 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.113.130:34819 (local address 127.15.113.190:44011)
0809 22:53:32.573567 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0809 22:53:32.573572 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:53:32.573589 (+ 17us) server_negotiation.cc:407] Connection header received
0809 22:53:32.573688 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:53:32.573693 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:53:32.573747 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:53:32.573823 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:53:32.574412 (+ 589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:32.575068 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:32.576206 (+ 1138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:32.576584 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:32.576698 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:32.580015 (+ 3317us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.113.130:34819: BlockingRecv error: recv got EOF from 127.15.113.130:34819 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
W20260809 22:53:32.580513 30441 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260809 22:53:32.890410 30446 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:53:32.890681 30446 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:53:32.890817 30446 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:53:32.897743 30446 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:53:32.897852 30446 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:53:32.898005 30446 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.113.131
I20260809 22:53:32.907263 30446 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.113.148:40757
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315935617919-15814-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315935617919-15814-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315935617919-15814-0/krb5kdc/oryx_127.15.113.131.keytab
--principal=oryx/127.15.113.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.113.131:38311
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315935617919-15814-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.113.131
--webserver_port=38449
--tserver_master_addrs=127.15.113.190:44011
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.113.131
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315935617919-15814-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:53:32.908500 30446 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:53:32.909791 30446 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:53:32.922096 30452 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:53:32.922204 30456 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:53:32.922096 30453 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:53:32.922865 30446 server_base.cc:1034] running on GCE node
I20260809 22:53:32.923395 30446 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:53:32.924212 30446 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:53:32.925406 30446 hybrid_clock.cc:648] HybridClock initialized: now 1786316012925354 us; error 48 us; skew 500 ppm
I20260809 22:53:32.925691 30446 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:53:32 dist-test-slave-ncp9 krb5kdc[29136](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316012, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.113.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:53:32.928650 30446 init.cc:377] Logged in from keytab as oryx/127.15.113.131@KRBTEST.COM (short username oryx)
I20260809 22:53:32.930083 30446 webserver.cc:460] Webserver started at http://127.15.113.131:38449/ using document root <none> and password file <none>
I20260809 22:53:32.930717 30446 fs_manager.cc:359] Metadata directory not provided
I20260809 22:53:32.930832 30446 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:53:32.935904 30446 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.003s
I20260809 22:53:32.938424 30469 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:53:32.939656 30446 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 22:53:32.939791 30446 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315935617919-15814-0/minicluster-data/ts-2/data,/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315935617919-15814-0/minicluster-data/ts-2/wal
uuid: "b3daf1cf1f124f749b4f928727aa586a"
format_stamp: "Formatted at 2026-08-09 22:53:30 on dist-test-slave-ncp9"
server_key: "70816cd73a69ebf7035dfea3396261b2"
server_key_iv: "f6811e9ef5aa99cad35860677034d145"
server_key_version: "encryptionkey@0"
I20260809 22:53:32.940738 30446 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315935617919-15814-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315935617919-15814-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315935617919-15814-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:53:32.952785 30446 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:53:32.956329 30446 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:53:32.956585 30446 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:53:32.957492 30446 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:53:32.958719 30446 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:53:32.958815 30446 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:53:32.958917 30446 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:53:32.958998 30446 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:53:32.988390 30446 rpc_server.cc:307] RPC server started. Bound to: 127.15.113.131:38311
I20260809 22:53:32.988456 30639 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.113.131:38311 every 8 connection(s)
I20260809 22:53:32.989952 30446 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315935617919-15814-0/minicluster-data/ts-2/data/info.pb
I20260809 22:53:32.999765 15814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskItYPJR/build/asan/bin/kudu as pid 30446
Aug 09 22:53:32 dist-test-slave-ncp9 krb5kdc[29136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.113.131@KRBTEST.COM for oryx/127.15.113.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 22:53:32 dist-test-slave-ncp9 krb5kdc[29136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.113.131@KRBTEST.COM for oryx/127.15.113.190@KRBTEST.COM, Server not found in Kerberos database
W20260809 22:53:33.005077 30255 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:53:32.992374 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.113.131:53563 (local address 127.15.113.190:44011)
0809 22:53:32.992591 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0809 22:53:32.992598 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:53:32.993737 (+ 1139us) server_negotiation.cc:407] Connection header received
0809 22:53:32.995460 (+ 1723us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:53:32.995464 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:53:32.995533 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:53:32.995618 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:53:32.997064 (+ 1446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:32.997872 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:32.999337 (+ 1465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:32.999754 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:32.999851 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:33.004798 (+ 4947us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.113.131:53563: BlockingRecv error: recv got EOF from 127.15.113.131:53563 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
I20260809 22:53:33.005584 30644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:32.992854 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:44011 (local address 127.15.113.131:53563)
0809 22:53:32.993583 (+ 729us) negotiation.cc:107] Waiting for socket to connect
0809 22:53:32.993627 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 22:53:32.995219 (+ 1592us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:53:32.995801 (+ 582us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:53:32.995808 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:53:32.996301 (+ 493us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:53:32.996884 (+ 583us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:53:32.996923 (+ 39us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:53:32.998006 (+ 1083us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:53:32.998010 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:53:32.999232 (+ 1222us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:53:32.999244 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:53:32.999985 (+ 741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:53:32.999993 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:53:33.000275 (+ 282us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:33.001047 (+ 772us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:53:33.001079 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:53:33.004578 (+ 3499us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.113.190:44011: Server oryx/127.15.113.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":492,"thread_start_us":273,"threads_started":1}
W20260809 22:53:33.006119 30641 heartbeater.cc:644] Failed to heartbeat to 127.15.113.190:44011 (0 consecutive failures): Not authorized: Failed to ping master at 127.15.113.190:44011: Client connection negotiation failed: client connection to 127.15.113.190:44011: Server oryx/127.15.113.190@KRBTEST.COM not found in Kerberos database
Aug 09 22:53:33 dist-test-slave-ncp9 krb5kdc[29136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.113.131@KRBTEST.COM for oryx/127.15.113.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 22:53:33 dist-test-slave-ncp9 krb5kdc[29136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.113.131@KRBTEST.COM for oryx/127.15.113.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 22:53:33.014554 30644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:33.006598 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:44011 (local address 127.15.113.131:39253)
0809 22:53:33.006749 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0809 22:53:33.006766 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 22:53:33.006854 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:53:33.007105 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:53:33.007109 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:53:33.007410 (+ 301us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:53:33.007533 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:53:33.007540 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:53:33.008470 (+ 930us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:53:33.008476 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:53:33.009842 (+ 1366us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:53:33.009854 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:53:33.010594 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:53:33.010600 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:53:33.010758 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:33.011246 (+ 488us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:53:33.011262 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:53:33.014312 (+ 3050us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.113.190:44011: Server oryx/127.15.113.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":37}
W20260809 22:53:33.014746 30255 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:53:33.006732 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.113.131:39253 (local address 127.15.113.190:44011)
0809 22:53:33.006858 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0809 22:53:33.006861 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 22:53:33.006879 (+ 18us) server_negotiation.cc:407] Connection header received
0809 22:53:33.006940 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:53:33.006943 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:53:33.006987 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:53:33.007071 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:53:33.007671 (+ 600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:33.008321 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:33.009966 (+ 1645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:33.010454 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:33.010556 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:33.014493 (+ 3937us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.113.131:39253: BlockingRecv error: recv got EOF from 127.15.113.131:39253 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
I20260809 22:53:33.016726 30647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:33.000791 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38938 (local address 127.15.113.190:44011)
0809 22:53:33.001592 (+ 801us) server_negotiation.cc:206] Beginning negotiation
0809 22:53:33.001597 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:53:33.001640 (+ 43us) server_negotiation.cc:407] Connection header received
0809 22:53:33.001684 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:53:33.001688 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:53:33.001740 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:53:33.001820 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:53:33.003639 (+ 1819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:33.004372 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:33.006345 (+ 1973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:33.006955 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:33.007057 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:33.009900 (+ 2843us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:53:33.009908 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:53:33.009922 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:53:33.009967 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:53:33.013282 (+ 3315us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:53:33.013905 (+ 623us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:53:33.013911 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:53:33.013917 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:53:33.014102 (+ 185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:53:33.014504 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:53:33.014508 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:53:33.014510 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:53:33.015145 (+ 635us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:53:33.015430 (+ 285us) server_negotiation.cc:1035] Waiting for connection context
0809 22:53:33.015990 (+ 560us) server_negotiation.cc:299] Negotiation successful
0809 22:53:33.016319 (+ 329us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":695,"thread_start_us":574,"threads_started":1}
Aug 09 22:53:33 dist-test-slave-ncp9 krb5kdc[29136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.113.131@KRBTEST.COM for oryx/127.15.113.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 22:53:33 dist-test-slave-ncp9 krb5kdc[29136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.113.131@KRBTEST.COM for oryx/127.15.113.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 22:53:33.023237 30644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:33.015233 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:44011 (local address 127.15.113.131:46139)
0809 22:53:33.015515 (+ 282us) negotiation.cc:107] Waiting for socket to connect
0809 22:53:33.015528 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 22:53:33.015608 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:53:33.015892 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:53:33.015897 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:53:33.016244 (+ 347us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:53:33.016414 (+ 170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:53:33.016425 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:53:33.017780 (+ 1355us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:53:33.017784 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:53:33.019030 (+ 1246us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:53:33.019042 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:53:33.019822 (+ 780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:53:33.019828 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:53:33.019985 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:33.020428 (+ 443us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:53:33.020443 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:53:33.022979 (+ 2536us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.113.190:44011: Server oryx/127.15.113.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":48}
W20260809 22:53:33.023481 30255 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:53:33.015290 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.113.131:46139 (local address 127.15.113.190:44011)
0809 22:53:33.015487 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0809 22:53:33.015491 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 22:53:33.015592 (+ 101us) server_negotiation.cc:407] Connection header received
0809 22:53:33.015684 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:53:33.015688 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:53:33.015771 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:53:33.015854 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:53:33.016545 (+ 691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:33.017654 (+ 1109us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:33.019156 (+ 1502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:33.019695 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:33.019795 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:33.023177 (+ 3382us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.113.131:46139: BlockingRecv error: recv got EOF from 127.15.113.131:46139 (error 108)
Metrics: {"server-negotiator.queue_time_us":112}
W20260809 22:53:33.023581 30641 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 09 22:53:33 dist-test-slave-ncp9 krb5kdc[29136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.113.129@KRBTEST.COM for oryx/127.15.113.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 22:53:33 dist-test-slave-ncp9 krb5kdc[29136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.113.129@KRBTEST.COM for oryx/127.15.113.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 22:53:33.155752 30648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:33.147966 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:44011 (local address 127.15.113.129:48715)
0809 22:53:33.148398 (+ 432us) negotiation.cc:107] Waiting for socket to connect
0809 22:53:33.148413 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 22:53:33.148522 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:53:33.148789 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:53:33.148792 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:53:33.149328 (+ 536us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:53:33.149523 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:53:33.149534 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:53:33.150466 (+ 932us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:53:33.150470 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:53:33.151402 (+ 932us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:53:33.151413 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:53:33.152037 (+ 624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:53:33.152042 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:53:33.152165 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:33.152720 (+ 555us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:53:33.152737 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:53:33.155440 (+ 2703us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.113.190:44011: Server oryx/127.15.113.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":327,"thread_start_us":214,"threads_started":1}
W20260809 22:53:33.155849 30255 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:53:33.148022 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.113.129:48715 (local address 127.15.113.190:44011)
0809 22:53:33.148241 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0809 22:53:33.148247 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:53:33.148484 (+ 237us) server_negotiation.cc:407] Connection header received
0809 22:53:33.148609 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:53:33.148613 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:53:33.148675 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:53:33.148750 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:53:33.149654 (+ 904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:33.150362 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:33.151501 (+ 1139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:33.151899 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:33.151984 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:33.155648 (+ 3664us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.113.129:48715: BlockingRecv error: recv got EOF from 127.15.113.129:48715 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
Aug 09 22:53:33 dist-test-slave-ncp9 krb5kdc[29136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.113.130@KRBTEST.COM for oryx/127.15.113.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 22:53:33 dist-test-slave-ncp9 krb5kdc[29136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.113.130@KRBTEST.COM for oryx/127.15.113.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 22:53:33.588951 30649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:33.581114 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:44011 (local address 127.15.113.130:52257)
0809 22:53:33.581567 (+ 453us) negotiation.cc:107] Waiting for socket to connect
0809 22:53:33.581584 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 22:53:33.581706 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:53:33.581981 (+ 275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:53:33.581984 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:53:33.582455 (+ 471us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:53:33.582657 (+ 202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:53:33.582666 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:53:33.583534 (+ 868us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:53:33.583538 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:53:33.584572 (+ 1034us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:53:33.584585 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:53:33.585385 (+ 800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:53:33.585390 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:53:33.585515 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:33.586033 (+ 518us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:53:33.586050 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:53:33.588691 (+ 2641us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.113.190:44011: Server oryx/127.15.113.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":346,"thread_start_us":226,"threads_started":1}
W20260809 22:53:33.589113 30255 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:53:33.581158 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.113.130:52257 (local address 127.15.113.190:44011)
0809 22:53:33.581378 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0809 22:53:33.581384 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:53:33.581670 (+ 286us) server_negotiation.cc:407] Connection header received
0809 22:53:33.581807 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:53:33.581810 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:53:33.581871 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:53:33.581941 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:53:33.582763 (+ 822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:33.583429 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:33.584712 (+ 1283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:33.585209 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:33.585310 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:33.588873 (+ 3563us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.113.130:52257: BlockingRecv error: recv got EOF from 127.15.113.130:52257 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Aug 09 22:53:34 dist-test-slave-ncp9 krb5kdc[29136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.113.131@KRBTEST.COM for oryx/127.15.113.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 22:53:34 dist-test-slave-ncp9 krb5kdc[29136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.113.131@KRBTEST.COM for oryx/127.15.113.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 22:53:34.034720 30650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:34.024106 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:44011 (local address 127.15.113.131:49255)
0809 22:53:34.024561 (+ 455us) negotiation.cc:107] Waiting for socket to connect
0809 22:53:34.024577 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 22:53:34.024690 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:53:34.024969 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:53:34.024972 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:53:34.025430 (+ 458us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:53:34.025623 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:53:34.025632 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:53:34.026533 (+ 901us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:53:34.026536 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:53:34.027485 (+ 949us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:53:34.027496 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:53:34.028088 (+ 592us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:53:34.028093 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:53:34.028217 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:34.031234 (+ 3017us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:53:34.031252 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:53:34.034415 (+ 3163us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.113.190:44011: Server oryx/127.15.113.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":345,"thread_start_us":225,"threads_started":1}
W20260809 22:53:34.034862 30255 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:53:34.024142 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.113.131:49255 (local address 127.15.113.190:44011)
0809 22:53:34.024327 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0809 22:53:34.024333 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:53:34.024658 (+ 325us) server_negotiation.cc:407] Connection header received
0809 22:53:34.024779 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:53:34.024782 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:53:34.024843 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:53:34.024951 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:53:34.025732 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:34.026426 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:34.027586 (+ 1160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:34.027960 (+ 374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:34.028043 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:34.034629 (+ 6586us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.113.131:49255: BlockingRecv error: recv got EOF from 127.15.113.131:49255 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
Aug 09 22:53:34 dist-test-slave-ncp9 krb5kdc[29136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.113.129@KRBTEST.COM for oryx/127.15.113.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 22:53:34 dist-test-slave-ncp9 krb5kdc[29136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.113.129@KRBTEST.COM for oryx/127.15.113.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 22:53:34.164448 30651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:34.156680 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:44011 (local address 127.15.113.129:34689)
0809 22:53:34.157179 (+ 499us) negotiation.cc:107] Waiting for socket to connect
0809 22:53:34.157196 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 22:53:34.157311 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:53:34.157566 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:53:34.157569 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:53:34.157976 (+ 407us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:53:34.158161 (+ 185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:53:34.158172 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:53:34.159092 (+ 920us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:53:34.159095 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:53:34.160143 (+ 1048us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:53:34.160158 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:53:34.160934 (+ 776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:53:34.160942 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:53:34.161055 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:34.161574 (+ 519us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:53:34.161591 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:53:34.164215 (+ 2624us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.113.190:44011: Server oryx/127.15.113.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":384,"thread_start_us":273,"threads_started":1}
W20260809 22:53:34.164599 30255 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:53:34.156751 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.113.129:34689 (local address 127.15.113.190:44011)
0809 22:53:34.156978 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0809 22:53:34.156984 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:53:34.157269 (+ 285us) server_negotiation.cc:407] Connection header received
0809 22:53:34.157397 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:53:34.157401 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:53:34.157462 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:53:34.157538 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:53:34.158265 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:34.158990 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:34.160277 (+ 1287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:34.160770 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:34.160859 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:34.164384 (+ 3525us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.113.129:34689: BlockingRecv error: recv got EOF from 127.15.113.129:34689 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
Aug 09 22:53:34 dist-test-slave-ncp9 krb5kdc[29136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.113.130@KRBTEST.COM for oryx/127.15.113.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 22:53:34 dist-test-slave-ncp9 krb5kdc[29136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.113.130@KRBTEST.COM for oryx/127.15.113.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 22:53:34.597880 30652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:34.589749 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:44011 (local address 127.15.113.130:52859)
0809 22:53:34.590273 (+ 524us) negotiation.cc:107] Waiting for socket to connect
0809 22:53:34.590289 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 22:53:34.590418 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:53:34.590722 (+ 304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:53:34.590727 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:53:34.591134 (+ 407us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:53:34.591318 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:53:34.591329 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:53:34.592331 (+ 1002us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:53:34.592335 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:53:34.593266 (+ 931us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:53:34.593279 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:53:34.593965 (+ 686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:53:34.593971 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:53:34.594119 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:34.594665 (+ 546us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:53:34.594681 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:53:34.597627 (+ 2946us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.113.190:44011: Server oryx/127.15.113.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":416,"thread_start_us":269,"threads_started":1}
W20260809 22:53:34.597987 30255 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:53:34.589870 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.113.130:52859 (local address 127.15.113.190:44011)
0809 22:53:34.590062 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0809 22:53:34.590069 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:53:34.590371 (+ 302us) server_negotiation.cc:407] Connection header received
0809 22:53:34.590527 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:53:34.590532 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:53:34.590592 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:53:34.590682 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:53:34.591472 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:34.592199 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:34.593397 (+ 1198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:34.593816 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:34.593919 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:34.597805 (+ 3886us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.113.130:52859: BlockingRecv error: recv got EOF from 127.15.113.130:52859 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
Aug 09 22:53:35 dist-test-slave-ncp9 krb5kdc[29136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.113.131@KRBTEST.COM for oryx/127.15.113.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 22:53:35 dist-test-slave-ncp9 krb5kdc[29136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.113.131@KRBTEST.COM for oryx/127.15.113.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 22:53:35.043638 30653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:35.035661 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:44011 (local address 127.15.113.131:46453)
0809 22:53:35.036169 (+ 508us) negotiation.cc:107] Waiting for socket to connect
0809 22:53:35.036185 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 22:53:35.036312 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:53:35.036620 (+ 308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:53:35.036625 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:53:35.037070 (+ 445us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:53:35.037244 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:53:35.037254 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:53:35.038257 (+ 1003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:53:35.038263 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:53:35.039165 (+ 902us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:53:35.039178 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:53:35.039870 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:53:35.039876 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:53:35.040036 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:35.040545 (+ 509us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:53:35.040561 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:53:35.043293 (+ 2732us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.113.190:44011: Server oryx/127.15.113.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":402,"thread_start_us":257,"threads_started":1}
W20260809 22:53:35.043798 30255 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:53:35.035805 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.113.131:46453 (local address 127.15.113.190:44011)
0809 22:53:35.036014 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0809 22:53:35.036020 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:53:35.036259 (+ 239us) server_negotiation.cc:407] Connection header received
0809 22:53:35.036425 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:53:35.036430 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:53:35.036493 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:53:35.036570 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:53:35.037383 (+ 813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:35.038108 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:35.039305 (+ 1197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:35.039722 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:35.039830 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:35.043513 (+ 3683us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.113.131:46453: BlockingRecv error: recv got EOF from 127.15.113.131:46453 (error 108)
Metrics: {"server-negotiator.queue_time_us":75}
Aug 09 22:53:35 dist-test-slave-ncp9 krb5kdc[29136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.113.129@KRBTEST.COM for oryx/127.15.113.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 22:53:35 dist-test-slave-ncp9 krb5kdc[29136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.113.129@KRBTEST.COM for oryx/127.15.113.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 22:53:35.173343 30655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:35.165275 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:44011 (local address 127.15.113.129:41355)
0809 22:53:35.165743 (+ 468us) negotiation.cc:107] Waiting for socket to connect
0809 22:53:35.165759 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 22:53:35.165885 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:53:35.166207 (+ 322us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:53:35.166212 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:53:35.166581 (+ 369us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:53:35.166746 (+ 165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:53:35.166757 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:53:35.167777 (+ 1020us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:53:35.167783 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:53:35.168696 (+ 913us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:53:35.168709 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:53:35.169427 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:53:35.169433 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:53:35.169604 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:35.170193 (+ 589us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:53:35.170211 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:53:35.173089 (+ 2878us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.113.190:44011: Server oryx/127.15.113.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":378,"thread_start_us":248,"threads_started":1}
W20260809 22:53:35.173475 30255 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:53:35.165365 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.113.129:41355 (local address 127.15.113.190:44011)
0809 22:53:35.165546 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0809 22:53:35.165551 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:53:35.165835 (+ 284us) server_negotiation.cc:407] Connection header received
0809 22:53:35.165999 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:53:35.166005 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:53:35.166072 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:53:35.166151 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:53:35.166894 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:35.167631 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:35.168833 (+ 1202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:35.169286 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:35.169395 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:35.173268 (+ 3873us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.113.129:41355: BlockingRecv error: recv got EOF from 127.15.113.129:41355 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
Aug 09 22:53:35 dist-test-slave-ncp9 krb5kdc[29136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.113.130@KRBTEST.COM for oryx/127.15.113.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 22:53:35 dist-test-slave-ncp9 krb5kdc[29136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.113.130@KRBTEST.COM for oryx/127.15.113.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 22:53:35.606586 30656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:35.598652 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:44011 (local address 127.15.113.130:42253)
0809 22:53:35.599135 (+ 483us) negotiation.cc:107] Waiting for socket to connect
0809 22:53:35.599163 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0809 22:53:35.599293 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:53:35.599592 (+ 299us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:53:35.599597 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:53:35.599965 (+ 368us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:53:35.600133 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:53:35.600142 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:53:35.601213 (+ 1071us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:53:35.601219 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:53:35.602107 (+ 888us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:53:35.602118 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:53:35.602843 (+ 725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:53:35.602849 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:53:35.603015 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:35.603548 (+ 533us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:53:35.603564 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:53:35.606270 (+ 2706us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.113.190:44011: Server oryx/127.15.113.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":391,"thread_start_us":277,"threads_started":1}
W20260809 22:53:35.606698 30255 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:53:35.598752 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.113.130:42253 (local address 127.15.113.190:44011)
0809 22:53:35.598948 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0809 22:53:35.598954 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:53:35.599245 (+ 291us) server_negotiation.cc:407] Connection header received
0809 22:53:35.599404 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:53:35.599408 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:53:35.599467 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:53:35.599546 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:53:35.600284 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:35.601089 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:35.602271 (+ 1182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:35.602708 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:35.602813 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:35.606439 (+ 3626us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.113.130:42253: BlockingRecv error: recv got EOF from 127.15.113.130:42253 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
Aug 09 22:53:36 dist-test-slave-ncp9 krb5kdc[29136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.113.131@KRBTEST.COM for oryx/127.15.113.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 22:53:36 dist-test-slave-ncp9 krb5kdc[29136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.113.131@KRBTEST.COM for oryx/127.15.113.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 22:53:36.052222 30657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:36.044672 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:44011 (local address 127.15.113.131:40605)
0809 22:53:36.045226 (+ 554us) negotiation.cc:107] Waiting for socket to connect
0809 22:53:36.045246 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 22:53:36.045363 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:53:36.045637 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:53:36.045640 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:53:36.045991 (+ 351us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:53:36.046173 (+ 182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:53:36.046183 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:53:36.047105 (+ 922us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:53:36.047108 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:53:36.048018 (+ 910us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:53:36.048031 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:53:36.048616 (+ 585us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:53:36.048620 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:53:36.048738 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:36.049338 (+ 600us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:53:36.049356 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:53:36.051957 (+ 2601us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.113.190:44011: Server oryx/127.15.113.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":432,"thread_start_us":306,"threads_started":1}
W20260809 22:53:36.052318 30255 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:53:36.044719 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.113.131:40605 (local address 127.15.113.190:44011)
0809 22:53:36.044883 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0809 22:53:36.044917 (+ 34us) server_negotiation.cc:399] Waiting for connection header
0809 22:53:36.045328 (+ 411us) server_negotiation.cc:407] Connection header received
0809 22:53:36.045447 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:53:36.045450 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:53:36.045528 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:53:36.045608 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:53:36.046305 (+ 697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:36.047012 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:36.048123 (+ 1111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:36.048511 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:36.048604 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:36.052134 (+ 3530us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.113.131:40605: BlockingRecv error: recv got EOF from 127.15.113.131:40605 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
Aug 09 22:53:36 dist-test-slave-ncp9 krb5kdc[29136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.113.129@KRBTEST.COM for oryx/127.15.113.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 22:53:36 dist-test-slave-ncp9 krb5kdc[29136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.113.129@KRBTEST.COM for oryx/127.15.113.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 22:53:36.182207 30658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:36.174219 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:44011 (local address 127.15.113.129:34123)
0809 22:53:36.174743 (+ 524us) negotiation.cc:107] Waiting for socket to connect
0809 22:53:36.174759 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 22:53:36.174876 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:53:36.175152 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:53:36.175156 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:53:36.175534 (+ 378us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:53:36.175688 (+ 154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:53:36.175698 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:53:36.176700 (+ 1002us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:53:36.176705 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:53:36.177705 (+ 1000us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:53:36.177719 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:53:36.178354 (+ 635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:53:36.178358 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:53:36.178483 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:36.179062 (+ 579us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:53:36.179080 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:53:36.181896 (+ 2816us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.113.190:44011: Server oryx/127.15.113.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":407,"thread_start_us":286,"threads_started":1}
W20260809 22:53:36.182406 30255 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:53:36.174276 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.113.129:34123 (local address 127.15.113.190:44011)
0809 22:53:36.174454 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0809 22:53:36.174458 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 22:53:36.174822 (+ 364us) server_negotiation.cc:407] Connection header received
0809 22:53:36.174977 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:53:36.174980 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:53:36.175039 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:53:36.175107 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:53:36.175793 (+ 686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:36.176562 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:36.177822 (+ 1260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:36.178236 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:36.178325 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:36.182156 (+ 3831us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.113.129:34123: BlockingRecv error: recv got EOF from 127.15.113.129:34123 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
Aug 09 22:53:36 dist-test-slave-ncp9 krb5kdc[29136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.113.130@KRBTEST.COM for oryx/127.15.113.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 22:53:36 dist-test-slave-ncp9 krb5kdc[29136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.113.130@KRBTEST.COM for oryx/127.15.113.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 22:53:36.615324 30659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:36.607530 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:44011 (local address 127.15.113.130:42307)
0809 22:53:36.607991 (+ 461us) negotiation.cc:107] Waiting for socket to connect
0809 22:53:36.608007 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 22:53:36.608123 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:53:36.608433 (+ 310us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:53:36.608436 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:53:36.608791 (+ 355us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:53:36.608994 (+ 203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:53:36.609005 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:53:36.609984 (+ 979us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:53:36.609990 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:53:36.611059 (+ 1069us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:53:36.611074 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:53:36.611758 (+ 684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:53:36.611763 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:53:36.611885 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:36.612430 (+ 545us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:53:36.612447 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:53:36.615098 (+ 2651us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.113.190:44011: Server oryx/127.15.113.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":350,"thread_start_us":242,"threads_started":1}
W20260809 22:53:36.615473 30255 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:53:36.607557 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.113.130:42307 (local address 127.15.113.190:44011)
0809 22:53:36.607758 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0809 22:53:36.607765 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:53:36.608065 (+ 300us) server_negotiation.cc:407] Connection header received
0809 22:53:36.608255 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:53:36.608260 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:53:36.608332 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:53:36.608410 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:53:36.609101 (+ 691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:36.609818 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:36.611183 (+ 1365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:36.611636 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:36.611733 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:36.615274 (+ 3541us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.113.130:42307: BlockingRecv error: recv got EOF from 127.15.113.130:42307 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
Aug 09 22:53:37 dist-test-slave-ncp9 krb5kdc[29136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.113.131@KRBTEST.COM for oryx/127.15.113.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 22:53:37 dist-test-slave-ncp9 krb5kdc[29136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.113.131@KRBTEST.COM for oryx/127.15.113.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 22:53:37.061136 30660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:37.053067 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:44011 (local address 127.15.113.131:49167)
0809 22:53:37.053545 (+ 478us) negotiation.cc:107] Waiting for socket to connect
0809 22:53:37.053561 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 22:53:37.053683 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:53:37.053945 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:53:37.053948 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:53:37.054316 (+ 368us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:53:37.054463 (+ 147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:53:37.054471 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:53:37.055670 (+ 1199us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:53:37.055676 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:53:37.056795 (+ 1119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:53:37.056811 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:53:37.057527 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:53:37.057534 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:53:37.057709 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:37.058236 (+ 527us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:53:37.058252 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:53:37.060861 (+ 2609us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.113.190:44011: Server oryx/127.15.113.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":372,"thread_start_us":265,"threads_started":1}
W20260809 22:53:37.061290 30255 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:53:37.053112 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.113.131:49167 (local address 127.15.113.190:44011)
0809 22:53:37.053265 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0809 22:53:37.053269 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 22:53:37.053621 (+ 352us) server_negotiation.cc:407] Connection header received
0809 22:53:37.053783 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:53:37.053787 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:53:37.053847 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:53:37.053918 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:53:37.054568 (+ 650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:37.055512 (+ 944us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:37.056974 (+ 1462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:37.057377 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:37.057504 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:37.061076 (+ 3572us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.113.131:49167: BlockingRecv error: recv got EOF from 127.15.113.131:49167 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
Aug 09 22:53:37 dist-test-slave-ncp9 krb5kdc[29136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.113.129@KRBTEST.COM for oryx/127.15.113.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 22:53:37 dist-test-slave-ncp9 krb5kdc[29136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.113.129@KRBTEST.COM for oryx/127.15.113.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 22:53:37.190649 30661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:37.183142 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:44011 (local address 127.15.113.129:44193)
0809 22:53:37.183643 (+ 501us) negotiation.cc:107] Waiting for socket to connect
0809 22:53:37.183661 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 22:53:37.183769 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:53:37.184019 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:53:37.184022 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:53:37.184412 (+ 390us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:53:37.184580 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:53:37.184589 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:53:37.185539 (+ 950us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:53:37.185542 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:53:37.186449 (+ 907us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:53:37.186461 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:53:37.187063 (+ 602us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:53:37.187067 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:53:37.187190 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:37.187751 (+ 561us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:53:37.187768 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:53:37.190405 (+ 2637us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.113.190:44011: Server oryx/127.15.113.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":393,"thread_start_us":240,"threads_started":1}
W20260809 22:53:37.190771 30255 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:53:37.183233 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.113.129:44193 (local address 127.15.113.190:44011)
0809 22:53:37.183410 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0809 22:53:37.183415 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:53:37.183725 (+ 310us) server_negotiation.cc:407] Connection header received
0809 22:53:37.183850 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:53:37.183853 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:53:37.183913 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:53:37.183990 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:53:37.184704 (+ 714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:37.185436 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:37.186553 (+ 1117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:37.186964 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:37.187050 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:37.190588 (+ 3538us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.113.129:44193: BlockingRecv error: recv got EOF from 127.15.113.129:44193 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
Aug 09 22:53:37 dist-test-slave-ncp9 krb5kdc[29136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.113.130@KRBTEST.COM for oryx/127.15.113.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 22:53:37 dist-test-slave-ncp9 krb5kdc[29136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.113.130@KRBTEST.COM for oryx/127.15.113.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 22:53:37.624284 30662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:37.616281 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:44011 (local address 127.15.113.130:38267)
0809 22:53:37.616764 (+ 483us) negotiation.cc:107] Waiting for socket to connect
0809 22:53:37.616779 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 22:53:37.616908 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:53:37.617249 (+ 341us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:53:37.617254 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:53:37.617690 (+ 436us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:53:37.617850 (+ 160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:53:37.617860 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:53:37.618891 (+ 1031us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:53:37.618897 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:53:37.619816 (+ 919us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:53:37.619828 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:53:37.620524 (+ 696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:53:37.620530 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:53:37.620705 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:37.621261 (+ 556us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:53:37.621278 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:53:37.624022 (+ 2744us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.113.190:44011: Server oryx/127.15.113.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":375,"thread_start_us":232,"threads_started":1}
W20260809 22:53:37.624437 30255 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:53:37.616452 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.113.130:38267 (local address 127.15.113.190:44011)
0809 22:53:37.616644 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0809 22:53:37.616650 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:53:37.616851 (+ 201us) server_negotiation.cc:407] Connection header received
0809 22:53:37.617025 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:53:37.617029 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:53:37.617091 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:53:37.617195 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:53:37.617997 (+ 802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:37.618745 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:37.619957 (+ 1212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:37.620386 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:37.620493 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:37.624221 (+ 3728us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.113.130:38267: BlockingRecv error: recv got EOF from 127.15.113.130:38267 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
Aug 09 22:53:38 dist-test-slave-ncp9 krb5kdc[29136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.113.131@KRBTEST.COM for oryx/127.15.113.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 22:53:38 dist-test-slave-ncp9 krb5kdc[29136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.113.131@KRBTEST.COM for oryx/127.15.113.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 22:53:38.069844 30663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:38.062045 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:44011 (local address 127.15.113.131:60507)
0809 22:53:38.062522 (+ 477us) negotiation.cc:107] Waiting for socket to connect
0809 22:53:38.062538 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 22:53:38.062649 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:53:38.062919 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:53:38.062922 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:53:38.063327 (+ 405us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:53:38.063495 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:53:38.063504 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:53:38.064416 (+ 912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:53:38.064419 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:53:38.065417 (+ 998us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:53:38.065431 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:53:38.066030 (+ 599us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:53:38.066034 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:53:38.066152 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:38.066762 (+ 610us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:53:38.066780 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:53:38.069483 (+ 2703us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.113.190:44011: Server oryx/127.15.113.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":370,"thread_start_us":220,"threads_started":1}
W20260809 22:53:38.069933 30255 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:53:38.062151 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.113.131:60507 (local address 127.15.113.190:44011)
0809 22:53:38.062381 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0809 22:53:38.062387 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:53:38.062607 (+ 220us) server_negotiation.cc:407] Connection header received
0809 22:53:38.062738 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:53:38.062742 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:53:38.062806 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:53:38.062884 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:53:38.063606 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:38.064303 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:38.065522 (+ 1219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:38.065913 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:38.065997 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:38.069706 (+ 3709us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.113.131:60507: BlockingRecv error: recv got EOF from 127.15.113.131:60507 (error 108)
Metrics: {"server-negotiator.queue_time_us":94}
Aug 09 22:53:38 dist-test-slave-ncp9 krb5kdc[29136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.113.129@KRBTEST.COM for oryx/127.15.113.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 22:53:38 dist-test-slave-ncp9 krb5kdc[29136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.113.129@KRBTEST.COM for oryx/127.15.113.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 22:53:38.199734 30664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:38.191491 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:44011 (local address 127.15.113.129:46543)
0809 22:53:38.191974 (+ 483us) negotiation.cc:107] Waiting for socket to connect
0809 22:53:38.191991 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 22:53:38.192107 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:53:38.192351 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:53:38.192354 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:53:38.192751 (+ 397us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:53:38.192936 (+ 185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:53:38.192946 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:53:38.193905 (+ 959us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:53:38.193909 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:53:38.194837 (+ 928us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:53:38.194850 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:53:38.195731 (+ 881us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:53:38.195738 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:53:38.195925 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:38.196615 (+ 690us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:53:38.196640 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:53:38.199517 (+ 2877us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.113.190:44011: Server oryx/127.15.113.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":372,"thread_start_us":253,"threads_started":1}
W20260809 22:53:38.199942 30255 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:53:38.191581 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.113.129:46543 (local address 127.15.113.190:44011)
0809 22:53:38.191814 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0809 22:53:38.191820 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:53:38.192061 (+ 241us) server_negotiation.cc:407] Connection header received
0809 22:53:38.192191 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:53:38.192195 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:53:38.192258 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:53:38.192334 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:53:38.193053 (+ 719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:38.193797 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:38.194957 (+ 1160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:38.195571 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:38.195674 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:38.199709 (+ 4035us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.113.129:46543: BlockingRecv error: recv got EOF from 127.15.113.129:46543 (error 108)
Metrics: {"server-negotiator.queue_time_us":75}
Aug 09 22:53:38 dist-test-slave-ncp9 krb5kdc[29136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.113.130@KRBTEST.COM for oryx/127.15.113.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 22:53:38 dist-test-slave-ncp9 krb5kdc[29136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.113.130@KRBTEST.COM for oryx/127.15.113.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 22:53:38.633100 30666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:38.625187 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:44011 (local address 127.15.113.130:42425)
0809 22:53:38.625843 (+ 656us) negotiation.cc:107] Waiting for socket to connect
0809 22:53:38.625860 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 22:53:38.625972 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:53:38.626231 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:53:38.626234 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:53:38.626633 (+ 399us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:53:38.626790 (+ 157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:53:38.626798 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:53:38.627756 (+ 958us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:53:38.627760 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:53:38.628654 (+ 894us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:53:38.628665 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:53:38.629359 (+ 694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:53:38.629364 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:53:38.629489 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:38.630069 (+ 580us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:53:38.630087 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:53:38.632840 (+ 2753us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.113.190:44011: Server oryx/127.15.113.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":552,"thread_start_us":424,"threads_started":1}
W20260809 22:53:38.633324 30255 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:53:38.625266 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.113.130:42425 (local address 127.15.113.190:44011)
0809 22:53:38.625526 (+ 260us) server_negotiation.cc:206] Beginning negotiation
0809 22:53:38.625533 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:53:38.625934 (+ 401us) server_negotiation.cc:407] Connection header received
0809 22:53:38.626062 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:53:38.626066 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:53:38.626128 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:53:38.626205 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:53:38.626927 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:38.627643 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:38.628790 (+ 1147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:38.629235 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:38.629331 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:38.633093 (+ 3762us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.113.130:42425: BlockingRecv error: recv got EOF from 127.15.113.130:42425 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
Aug 09 22:53:39 dist-test-slave-ncp9 krb5kdc[29136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.113.131@KRBTEST.COM for oryx/127.15.113.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 22:53:39 dist-test-slave-ncp9 krb5kdc[29136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.113.131@KRBTEST.COM for oryx/127.15.113.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 22:53:39.081805 30667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:39.070711 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:44011 (local address 127.15.113.131:58723)
0809 22:53:39.071203 (+ 492us) negotiation.cc:107] Waiting for socket to connect
0809 22:53:39.071218 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 22:53:39.071348 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:53:39.071671 (+ 323us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:53:39.071676 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:53:39.072082 (+ 406us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:53:39.072244 (+ 162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:53:39.072255 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:53:39.073367 (+ 1112us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:53:39.073371 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:53:39.074299 (+ 928us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:53:39.074312 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:53:39.075061 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:53:39.075067 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:53:39.075228 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:39.078570 (+ 3342us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:53:39.078587 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:53:39.081519 (+ 2932us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.113.190:44011: Server oryx/127.15.113.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":397,"thread_start_us":265,"threads_started":1}
W20260809 22:53:39.081933 30255 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:53:39.070780 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.113.131:58723 (local address 127.15.113.190:44011)
0809 22:53:39.070973 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0809 22:53:39.070980 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:53:39.071299 (+ 319us) server_negotiation.cc:407] Connection header received
0809 22:53:39.071457 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:53:39.071463 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:53:39.071534 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:53:39.071620 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:53:39.072386 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:39.073231 (+ 845us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:39.074443 (+ 1212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:39.074895 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:39.075019 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:39.081703 (+ 6684us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.113.131:58723: BlockingRecv error: recv got EOF from 127.15.113.131:58723 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
Aug 09 22:53:39 dist-test-slave-ncp9 krb5kdc[29136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.113.129@KRBTEST.COM for oryx/127.15.113.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 22:53:39 dist-test-slave-ncp9 krb5kdc[29136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.113.129@KRBTEST.COM for oryx/127.15.113.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 22:53:39.208595 30668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:39.200662 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:44011 (local address 127.15.113.129:39781)
0809 22:53:39.201267 (+ 605us) negotiation.cc:107] Waiting for socket to connect
0809 22:53:39.201283 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 22:53:39.201396 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:53:39.201660 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:53:39.201664 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:53:39.202060 (+ 396us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:53:39.202257 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:53:39.202267 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:53:39.203203 (+ 936us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:53:39.203206 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:53:39.204107 (+ 901us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:53:39.204128 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:53:39.204772 (+ 644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:53:39.204777 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:53:39.204883 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:39.205509 (+ 626us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:53:39.205525 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:53:39.208341 (+ 2816us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.113.190:44011: Server oryx/127.15.113.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":497,"thread_start_us":268,"threads_started":1}
W20260809 22:53:39.208698 30255 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:53:39.200728 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.113.129:39781 (local address 127.15.113.190:44011)
0809 22:53:39.200985 (+ 257us) server_negotiation.cc:206] Beginning negotiation
0809 22:53:39.200990 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:53:39.201350 (+ 360us) server_negotiation.cc:407] Connection header received
0809 22:53:39.201486 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:53:39.201490 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:53:39.201551 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:53:39.201631 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:53:39.202371 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:39.203091 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:39.204242 (+ 1151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:39.204661 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:39.204750 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:39.208520 (+ 3770us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.113.129:39781: BlockingRecv error: recv got EOF from 127.15.113.129:39781 (error 108)
Metrics: {"server-negotiator.queue_time_us":81}
Aug 09 22:53:39 dist-test-slave-ncp9 krb5kdc[29136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.113.130@KRBTEST.COM for oryx/127.15.113.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 22:53:39 dist-test-slave-ncp9 krb5kdc[29136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.113.130@KRBTEST.COM for oryx/127.15.113.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 22:53:39.642136 30672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:39.633991 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:44011 (local address 127.15.113.130:38491)
0809 22:53:39.634576 (+ 585us) negotiation.cc:107] Waiting for socket to connect
0809 22:53:39.634593 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 22:53:39.634713 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:53:39.634990 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:53:39.634993 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:53:39.635396 (+ 403us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:53:39.635578 (+ 182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:53:39.635591 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:53:39.636873 (+ 1282us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:53:39.636879 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:53:39.637853 (+ 974us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:53:39.637867 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:53:39.638494 (+ 627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:53:39.638499 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:53:39.638611 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:39.639139 (+ 528us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:53:39.639156 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:53:39.641877 (+ 2721us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.113.190:44011: Server oryx/127.15.113.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":472,"thread_start_us":252,"threads_started":1}
W20260809 22:53:39.642277 30255 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:53:39.634054 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.113.130:38491 (local address 127.15.113.190:44011)
0809 22:53:39.634219 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0809 22:53:39.634224 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:53:39.634654 (+ 430us) server_negotiation.cc:407] Connection header received
0809 22:53:39.634796 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:53:39.634799 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:53:39.634877 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:53:39.634961 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:53:39.635778 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:39.636725 (+ 947us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:39.637969 (+ 1244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:39.638380 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:39.638469 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:39.642049 (+ 3580us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.113.130:38491: BlockingRecv error: recv got EOF from 127.15.113.130:38491 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Aug 09 22:53:40 dist-test-slave-ncp9 krb5kdc[29136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.113.131@KRBTEST.COM for oryx/127.15.113.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 22:53:40 dist-test-slave-ncp9 krb5kdc[29136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.113.131@KRBTEST.COM for oryx/127.15.113.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 22:53:40.091007 30673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:40.082875 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:44011 (local address 127.15.113.131:55147)
0809 22:53:40.083443 (+ 568us) negotiation.cc:107] Waiting for socket to connect
0809 22:53:40.083459 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 22:53:40.083593 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:53:40.083898 (+ 305us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:53:40.083903 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:53:40.084350 (+ 447us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:53:40.084526 (+ 176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:53:40.084536 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:53:40.085658 (+ 1122us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:53:40.085663 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:53:40.086732 (+ 1069us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:53:40.086746 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:53:40.087454 (+ 708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:53:40.087461 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:53:40.087614 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:40.088149 (+ 535us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:53:40.088166 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:53:40.090755 (+ 2589us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.113.190:44011: Server oryx/127.15.113.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":466,"thread_start_us":241,"threads_started":1}
W20260809 22:53:40.091114 30255 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:53:40.082922 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.113.131:55147 (local address 127.15.113.190:44011)
0809 22:53:40.083090 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0809 22:53:40.083094 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 22:53:40.083546 (+ 452us) server_negotiation.cc:407] Connection header received
0809 22:53:40.083708 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:53:40.083713 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:53:40.083774 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:53:40.083855 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:53:40.084678 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:40.085522 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:40.086868 (+ 1346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:40.087305 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:40.087418 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:40.090941 (+ 3523us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.113.131:55147: BlockingRecv error: recv got EOF from 127.15.113.131:55147 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Aug 09 22:53:40 dist-test-slave-ncp9 krb5kdc[29136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.113.129@KRBTEST.COM for oryx/127.15.113.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 22:53:40 dist-test-slave-ncp9 krb5kdc[29136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.113.129@KRBTEST.COM for oryx/127.15.113.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 22:53:40.217451 30675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:40.209462 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:44011 (local address 127.15.113.129:47849)
0809 22:53:40.209966 (+ 504us) negotiation.cc:107] Waiting for socket to connect
0809 22:53:40.209983 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 22:53:40.210103 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:53:40.210377 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:53:40.210380 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:53:40.210820 (+ 440us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:53:40.210982 (+ 162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:53:40.210993 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:53:40.212132 (+ 1139us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:53:40.212139 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:53:40.213103 (+ 964us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:53:40.213119 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:53:40.213878 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:53:40.213885 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:53:40.214025 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:40.214567 (+ 542us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:53:40.214585 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:53:40.217146 (+ 2561us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.113.190:44011: Server oryx/127.15.113.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":382,"thread_start_us":257,"threads_started":1}
W20260809 22:53:40.217643 30255 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:53:40.209602 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.113.129:47849 (local address 127.15.113.190:44011)
0809 22:53:40.209840 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0809 22:53:40.209845 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:53:40.210058 (+ 213us) server_negotiation.cc:407] Connection header received
0809 22:53:40.210206 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:53:40.210210 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:53:40.210270 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:53:40.210342 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:53:40.211095 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:40.211974 (+ 879us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:40.213250 (+ 1276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:40.213686 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:40.213795 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:40.217389 (+ 3594us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.113.129:47849: BlockingRecv error: recv got EOF from 127.15.113.129:47849 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
Aug 09 22:53:40 dist-test-slave-ncp9 krb5kdc[29136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.113.130@KRBTEST.COM for oryx/127.15.113.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 22:53:40 dist-test-slave-ncp9 krb5kdc[29136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.113.130@KRBTEST.COM for oryx/127.15.113.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 22:53:40.651269 30676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:40.643023 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:44011 (local address 127.15.113.130:42993)
0809 22:53:40.643511 (+ 488us) negotiation.cc:107] Waiting for socket to connect
0809 22:53:40.643527 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 22:53:40.643641 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:53:40.643931 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:53:40.643935 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:53:40.644352 (+ 417us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:53:40.644535 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:53:40.644546 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:53:40.645746 (+ 1200us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:53:40.645750 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:53:40.646701 (+ 951us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:53:40.646713 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:53:40.647371 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:53:40.647375 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:53:40.647481 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:40.648046 (+ 565us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:53:40.648064 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:53:40.651032 (+ 2968us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.113.190:44011: Server oryx/127.15.113.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":360,"thread_start_us":250,"threads_started":1}
W20260809 22:53:40.651387 30255 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:53:40.643080 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.113.130:42993 (local address 127.15.113.190:44011)
0809 22:53:40.643279 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0809 22:53:40.643286 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:53:40.643602 (+ 316us) server_negotiation.cc:407] Connection header received
0809 22:53:40.643734 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:53:40.643737 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:53:40.643799 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:53:40.643875 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:53:40.644656 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:40.645612 (+ 956us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:40.646806 (+ 1194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:40.647239 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:40.647328 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:40.651213 (+ 3885us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.113.130:42993: BlockingRecv error: recv got EOF from 127.15.113.130:42993 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Aug 09 22:53:41 dist-test-slave-ncp9 krb5kdc[29136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.113.131@KRBTEST.COM for oryx/127.15.113.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 22:53:41 dist-test-slave-ncp9 krb5kdc[29136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.113.131@KRBTEST.COM for oryx/127.15.113.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 22:53:41.099884 30677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:41.091867 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:44011 (local address 127.15.113.131:43023)
0809 22:53:41.092340 (+ 473us) negotiation.cc:107] Waiting for socket to connect
0809 22:53:41.092358 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 22:53:41.092504 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:53:41.092762 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:53:41.092765 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:53:41.093202 (+ 437us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:53:41.093357 (+ 155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:53:41.093366 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:53:41.094293 (+ 927us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:53:41.094296 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:53:41.095257 (+ 961us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:53:41.095269 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:53:41.095898 (+ 629us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:53:41.095902 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:53:41.096023 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:41.096597 (+ 574us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:53:41.096615 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:53:41.099659 (+ 3044us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.113.190:44011: Server oryx/127.15.113.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":349,"thread_start_us":231,"threads_started":1}
W20260809 22:53:41.100059 30255 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:53:41.091983 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.113.131:43023 (local address 127.15.113.190:44011)
0809 22:53:41.092226 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0809 22:53:41.092232 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:53:41.092427 (+ 195us) server_negotiation.cc:407] Connection header received
0809 22:53:41.092596 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:53:41.092600 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:53:41.092661 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:53:41.092737 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:53:41.093461 (+ 724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:41.094180 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:41.095367 (+ 1187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:41.095763 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:41.095851 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:41.099848 (+ 3997us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.113.131:43023: BlockingRecv error: recv got EOF from 127.15.113.131:43023 (error 108)
Metrics: {"server-negotiator.queue_time_us":83}
Aug 09 22:53:41 dist-test-slave-ncp9 krb5kdc[29136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.113.129@KRBTEST.COM for oryx/127.15.113.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 22:53:41 dist-test-slave-ncp9 krb5kdc[29136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.113.129@KRBTEST.COM for oryx/127.15.113.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 22:53:41.226392 30679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:41.218296 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:44011 (local address 127.15.113.129:44965)
0809 22:53:41.218800 (+ 504us) negotiation.cc:107] Waiting for socket to connect
0809 22:53:41.218815 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 22:53:41.218926 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:53:41.219195 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:53:41.219198 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:53:41.219581 (+ 383us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:53:41.219747 (+ 166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:53:41.219756 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:53:41.220710 (+ 954us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:53:41.220713 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:53:41.221773 (+ 1060us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:53:41.221787 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:53:41.222544 (+ 757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:53:41.222551 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:53:41.222691 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:41.223247 (+ 556us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:53:41.223264 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:53:41.226165 (+ 2901us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.113.190:44011: Server oryx/127.15.113.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":404,"thread_start_us":263,"threads_started":1}
W20260809 22:53:41.226567 30255 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:53:41.218331 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.113.129:44965 (local address 127.15.113.190:44011)
0809 22:53:41.218568 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0809 22:53:41.218575 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:53:41.218888 (+ 313us) server_negotiation.cc:407] Connection header received
0809 22:53:41.219013 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:53:41.219017 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:53:41.219079 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:53:41.219155 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:53:41.219852 (+ 697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:41.220600 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:41.221903 (+ 1303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:41.222411 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:41.222505 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:41.226348 (+ 3843us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.113.129:44965: BlockingRecv error: recv got EOF from 127.15.113.129:44965 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
Aug 09 22:53:41 dist-test-slave-ncp9 krb5kdc[29136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.113.130@KRBTEST.COM for oryx/127.15.113.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 22:53:41 dist-test-slave-ncp9 krb5kdc[29136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.113.130@KRBTEST.COM for oryx/127.15.113.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 22:53:41.661648 30681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:41.652125 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:44011 (local address 127.15.113.130:41363)
0809 22:53:41.652698 (+ 573us) negotiation.cc:107] Waiting for socket to connect
0809 22:53:41.652719 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0809 22:53:41.652860 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:53:41.653240 (+ 380us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:53:41.653244 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:53:41.653755 (+ 511us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:53:41.653980 (+ 225us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:53:41.653994 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:53:41.655049 (+ 1055us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:53:41.655055 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:53:41.656555 (+ 1500us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:53:41.656589 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:53:41.657449 (+ 860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:53:41.657456 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:53:41.657637 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:41.658246 (+ 609us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:53:41.658270 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:53:41.661413 (+ 3143us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.113.190:44011: Server oryx/127.15.113.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":442,"thread_start_us":281,"threads_started":1}
W20260809 22:53:41.661772 30255 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:53:41.652197 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.113.130:41363 (local address 127.15.113.190:44011)
0809 22:53:41.652346 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0809 22:53:41.652351 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:53:41.652808 (+ 457us) server_negotiation.cc:407] Connection header received
0809 22:53:41.652991 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:53:41.652997 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:53:41.653078 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:53:41.653170 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:53:41.654168 (+ 998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:41.654927 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:41.656726 (+ 1799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:41.657287 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:41.657395 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:41.661577 (+ 4182us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.113.130:41363: BlockingRecv error: recv got EOF from 127.15.113.130:41363 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
Aug 09 22:53:42 dist-test-slave-ncp9 krb5kdc[29136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.113.131@KRBTEST.COM for oryx/127.15.113.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 22:53:42 dist-test-slave-ncp9 krb5kdc[29136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.113.131@KRBTEST.COM for oryx/127.15.113.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 22:53:42.108736 30683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:42.100743 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:44011 (local address 127.15.113.131:49343)
0809 22:53:42.101335 (+ 592us) negotiation.cc:107] Waiting for socket to connect
0809 22:53:42.101350 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 22:53:42.101460 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:53:42.101717 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:53:42.101720 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:53:42.102101 (+ 381us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:53:42.102281 (+ 180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:53:42.102291 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:53:42.103340 (+ 1049us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:53:42.103345 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:53:42.104587 (+ 1242us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:53:42.104600 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:53:42.105301 (+ 701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:53:42.105306 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:53:42.105425 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:42.105940 (+ 515us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:53:42.105956 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:53:42.108495 (+ 2539us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.113.190:44011: Server oryx/127.15.113.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":486,"thread_start_us":336,"threads_started":1}
W20260809 22:53:42.108870 30255 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:53:42.100810 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.113.131:49343 (local address 127.15.113.190:44011)
0809 22:53:42.100979 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0809 22:53:42.100983 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 22:53:42.101412 (+ 429us) server_negotiation.cc:407] Connection header received
0809 22:53:42.101546 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:53:42.101549 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:53:42.101608 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:53:42.101684 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:53:42.102386 (+ 702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:42.103208 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:42.104724 (+ 1516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:42.105184 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:42.105276 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:42.108671 (+ 3395us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.113.131:49343: BlockingRecv error: recv got EOF from 127.15.113.131:49343 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
Aug 09 22:53:42 dist-test-slave-ncp9 krb5kdc[29136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.113.129@KRBTEST.COM for oryx/127.15.113.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 22:53:42 dist-test-slave-ncp9 krb5kdc[29136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.113.129@KRBTEST.COM for oryx/127.15.113.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 22:53:42.235698 30684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:42.227305 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:44011 (local address 127.15.113.129:49235)
0809 22:53:42.227765 (+ 460us) negotiation.cc:107] Waiting for socket to connect
0809 22:53:42.227781 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 22:53:42.227889 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:53:42.228146 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:53:42.228150 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:53:42.228529 (+ 379us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:53:42.228684 (+ 155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:53:42.228694 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:53:42.229999 (+ 1305us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:53:42.230004 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:53:42.231016 (+ 1012us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:53:42.231030 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:53:42.231715 (+ 685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:53:42.231720 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:53:42.231823 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:42.232350 (+ 527us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:53:42.232366 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:53:42.235371 (+ 3005us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.113.190:44011: Server oryx/127.15.113.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":364,"thread_start_us":250,"threads_started":1}
W20260809 22:53:42.235808 30255 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:53:42.227373 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.113.129:49235 (local address 127.15.113.190:44011)
0809 22:53:42.227609 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0809 22:53:42.227616 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:53:42.227849 (+ 233us) server_negotiation.cc:407] Connection header received
0809 22:53:42.227974 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:53:42.227977 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:53:42.228038 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:53:42.228114 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:53:42.228808 (+ 694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:42.229855 (+ 1047us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:42.231146 (+ 1291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:42.231600 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:42.231693 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:42.235577 (+ 3884us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.113.129:49235: BlockingRecv error: recv got EOF from 127.15.113.129:49235 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
Aug 09 22:53:42 dist-test-slave-ncp9 krb5kdc[29136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.113.130@KRBTEST.COM for oryx/127.15.113.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 22:53:42 dist-test-slave-ncp9 krb5kdc[29136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.113.130@KRBTEST.COM for oryx/127.15.113.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 22:53:42.673491 30687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:42.662527 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:44011 (local address 127.15.113.130:58535)
0809 22:53:42.662997 (+ 470us) negotiation.cc:107] Waiting for socket to connect
0809 22:53:42.663013 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 22:53:42.663120 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:53:42.663376 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:53:42.663379 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:53:42.663734 (+ 355us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:53:42.663879 (+ 145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:53:42.663888 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:53:42.664793 (+ 905us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:53:42.664796 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:53:42.665784 (+ 988us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:53:42.665801 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:53:42.666682 (+ 881us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:53:42.666688 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:53:42.666817 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:42.670557 (+ 3740us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:53:42.670574 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:53:42.673270 (+ 2696us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.113.190:44011: Server oryx/127.15.113.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":362,"thread_start_us":258,"threads_started":1}
W20260809 22:53:42.673676 30255 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:53:42.662598 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.113.130:58535 (local address 127.15.113.190:44011)
0809 22:53:42.662748 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0809 22:53:42.662752 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 22:53:42.663074 (+ 322us) server_negotiation.cc:407] Connection header received
0809 22:53:42.663211 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:53:42.663215 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:53:42.663274 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:53:42.663349 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:53:42.663985 (+ 636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:42.664678 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:42.665923 (+ 1245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:42.666522 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:42.666646 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:42.673454 (+ 6808us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.113.130:58535: BlockingRecv error: recv got EOF from 127.15.113.130:58535 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
Aug 09 22:53:43 dist-test-slave-ncp9 krb5kdc[29136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.113.131@KRBTEST.COM for oryx/127.15.113.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 22:53:43 dist-test-slave-ncp9 krb5kdc[29136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.113.131@KRBTEST.COM for oryx/127.15.113.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 22:53:43.117725 30688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:43.109583 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:44011 (local address 127.15.113.131:59893)
0809 22:53:43.110060 (+ 477us) negotiation.cc:107] Waiting for socket to connect
0809 22:53:43.110078 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 22:53:43.110204 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:53:43.110566 (+ 362us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:53:43.110571 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:53:43.110940 (+ 369us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:53:43.111087 (+ 147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:53:43.111095 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:53:43.112376 (+ 1281us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:53:43.112381 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:53:43.113366 (+ 985us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:53:43.113379 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:53:43.114292 (+ 913us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:53:43.114296 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:53:43.114403 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:43.114925 (+ 522us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:53:43.114941 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:53:43.117503 (+ 2562us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.113.190:44011: Server oryx/127.15.113.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":376,"thread_start_us":259,"threads_started":1}
W20260809 22:53:43.117898 30255 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:53:43.109770 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.113.131:59893 (local address 127.15.113.190:44011)
0809 22:53:43.109984 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0809 22:53:43.109990 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:53:43.110164 (+ 174us) server_negotiation.cc:407] Connection header received
0809 22:53:43.110339 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:53:43.110344 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:53:43.110436 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:53:43.110513 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:53:43.111215 (+ 702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:43.112244 (+ 1029us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:43.113497 (+ 1253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:43.114167 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:43.114286 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:43.117690 (+ 3404us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.113.131:59893: BlockingRecv error: recv got EOF from 127.15.113.131:59893 (error 108)
Metrics: {"server-negotiator.queue_time_us":85}
Aug 09 22:53:43 dist-test-slave-ncp9 krb5kdc[29136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.113.129@KRBTEST.COM for oryx/127.15.113.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 22:53:43 dist-test-slave-ncp9 krb5kdc[29136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.113.129@KRBTEST.COM for oryx/127.15.113.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 22:53:43.244256 30689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:43.236647 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:44011 (local address 127.15.113.129:60139)
0809 22:53:43.237169 (+ 522us) negotiation.cc:107] Waiting for socket to connect
0809 22:53:43.237185 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 22:53:43.237299 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:53:43.237577 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:53:43.237579 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:53:43.237987 (+ 408us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:53:43.238191 (+ 204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:53:43.238202 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:53:43.239102 (+ 900us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:53:43.239105 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:53:43.239987 (+ 882us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:53:43.239999 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:53:43.240631 (+ 632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:53:43.240635 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:53:43.240758 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:43.241310 (+ 552us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:53:43.241326 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:53:43.243992 (+ 2666us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.113.190:44011: Server oryx/127.15.113.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":403,"thread_start_us":274,"threads_started":1}
W20260809 22:53:43.244452 30255 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:53:43.236692 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.113.129:60139 (local address 127.15.113.190:44011)
0809 22:53:43.236845 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0809 22:53:43.236850 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:53:43.237247 (+ 397us) server_negotiation.cc:407] Connection header received
0809 22:53:43.237411 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:53:43.237415 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:53:43.237485 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:53:43.237563 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:53:43.238300 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:43.239006 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:43.240087 (+ 1081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:43.240517 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:43.240600 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:43.244213 (+ 3613us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.113.129:60139: BlockingRecv error: recv got EOF from 127.15.113.129:60139 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
Aug 09 22:53:43 dist-test-slave-ncp9 krb5kdc[29136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.113.130@KRBTEST.COM for oryx/127.15.113.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 22:53:43 dist-test-slave-ncp9 krb5kdc[29136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.113.130@KRBTEST.COM for oryx/127.15.113.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 22:53:43.682116 30690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:43.674341 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:44011 (local address 127.15.113.130:43657)
0809 22:53:43.674841 (+ 500us) negotiation.cc:107] Waiting for socket to connect
0809 22:53:43.674858 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 22:53:43.674973 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:53:43.675214 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:53:43.675216 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:53:43.675613 (+ 397us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:53:43.675809 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:53:43.675819 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:53:43.676714 (+ 895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:53:43.676718 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:53:43.677711 (+ 993us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:53:43.677725 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:53:43.678403 (+ 678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:53:43.678407 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:53:43.678525 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:43.679048 (+ 523us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:53:43.679064 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:53:43.681794 (+ 2730us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.113.190:44011: Server oryx/127.15.113.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":388,"thread_start_us":264,"threads_started":1}
W20260809 22:53:43.682240 30255 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:53:43.674407 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.113.130:43657 (local address 127.15.113.190:44011)
0809 22:53:43.674562 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0809 22:53:43.674566 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 22:53:43.674921 (+ 355us) server_negotiation.cc:407] Connection header received
0809 22:53:43.675056 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:53:43.675060 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:53:43.675117 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:53:43.675200 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:53:43.675911 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:43.676602 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:43.677850 (+ 1248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:43.678291 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:43.678383 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:43.682010 (+ 3627us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.113.130:43657: BlockingRecv error: recv got EOF from 127.15.113.130:43657 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
Aug 09 22:53:44 dist-test-slave-ncp9 krb5kdc[29136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.113.131@KRBTEST.COM for oryx/127.15.113.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 22:53:44 dist-test-slave-ncp9 krb5kdc[29136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.113.131@KRBTEST.COM for oryx/127.15.113.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 22:53:44.130004 30692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:44.118581 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:44011 (local address 127.15.113.131:44537)
0809 22:53:44.119113 (+ 532us) negotiation.cc:107] Waiting for socket to connect
0809 22:53:44.119129 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 22:53:44.119237 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:53:44.119488 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:53:44.119491 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:53:44.119899 (+ 408us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:53:44.120079 (+ 180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:53:44.120088 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:53:44.121093 (+ 1005us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:53:44.121097 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:53:44.122078 (+ 981us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:53:44.122095 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:53:44.122975 (+ 880us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:53:44.122981 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:53:44.123169 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:44.127086 (+ 3917us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:53:44.127104 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:53:44.129674 (+ 2570us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.113.190:44011: Server oryx/127.15.113.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":426,"thread_start_us":301,"threads_started":1}
W20260809 22:53:44.130108 30255 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:53:44.118685 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.113.131:44537 (local address 127.15.113.190:44011)
0809 22:53:44.118850 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0809 22:53:44.118854 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 22:53:44.119193 (+ 339us) server_negotiation.cc:407] Connection header received
0809 22:53:44.119321 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:53:44.119325 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:53:44.119384 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:53:44.119461 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:53:44.120200 (+ 739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:44.120970 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:44.122250 (+ 1280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:44.122829 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:44.122950 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:44.129886 (+ 6936us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.113.131:44537: BlockingRecv error: recv got EOF from 127.15.113.131:44537 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
Aug 09 22:53:44 dist-test-slave-ncp9 krb5kdc[29136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.113.129@KRBTEST.COM for oryx/127.15.113.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 22:53:44 dist-test-slave-ncp9 krb5kdc[29136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.113.129@KRBTEST.COM for oryx/127.15.113.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 22:53:44.253358 30693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:44.245240 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:44011 (local address 127.15.113.129:53863)
0809 22:53:44.245765 (+ 525us) negotiation.cc:107] Waiting for socket to connect
0809 22:53:44.245781 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 22:53:44.245897 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:53:44.246212 (+ 315us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:53:44.246215 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:53:44.246602 (+ 387us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:53:44.246761 (+ 159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:53:44.246772 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:53:44.247743 (+ 971us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:53:44.247746 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:53:44.248655 (+ 909us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:53:44.248667 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:53:44.249506 (+ 839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:53:44.249513 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:53:44.249697 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:44.250328 (+ 631us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:53:44.250352 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:53:44.253116 (+ 2764us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.113.190:44011: Server oryx/127.15.113.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":415,"thread_start_us":297,"threads_started":1}
W20260809 22:53:44.253525 30255 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:53:44.245290 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.113.129:53863 (local address 127.15.113.190:44011)
0809 22:53:44.245445 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0809 22:53:44.245449 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 22:53:44.245845 (+ 396us) server_negotiation.cc:407] Connection header received
0809 22:53:44.246006 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:53:44.246011 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:53:44.246090 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:53:44.246172 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:53:44.246871 (+ 699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:44.247637 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:44.248786 (+ 1149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:44.249358 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:44.249469 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:44.253323 (+ 3854us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.113.129:53863: BlockingRecv error: recv got EOF from 127.15.113.129:53863 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
Aug 09 22:53:44 dist-test-slave-ncp9 krb5kdc[29136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.113.130@KRBTEST.COM for oryx/127.15.113.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 22:53:44 dist-test-slave-ncp9 krb5kdc[29136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.113.130@KRBTEST.COM for oryx/127.15.113.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 22:53:44.691747 30699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:44.683038 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:44011 (local address 127.15.113.130:56387)
0809 22:53:44.683640 (+ 602us) negotiation.cc:107] Waiting for socket to connect
0809 22:53:44.683658 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 22:53:44.683807 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:53:44.684171 (+ 364us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:53:44.684176 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:53:44.684647 (+ 471us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:53:44.684820 (+ 173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:53:44.684831 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:53:44.685875 (+ 1044us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:53:44.685880 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:53:44.686848 (+ 968us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:53:44.686861 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:53:44.687663 (+ 802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:53:44.687670 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:53:44.687866 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:44.688453 (+ 587us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:53:44.688473 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:53:44.691423 (+ 2950us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.113.190:44011: Server oryx/127.15.113.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":491,"thread_start_us":337,"threads_started":1}
W20260809 22:53:44.691877 30255 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:53:44.683099 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.113.130:56387 (local address 127.15.113.190:44011)
0809 22:53:44.683388 (+ 289us) server_negotiation.cc:206] Beginning negotiation
0809 22:53:44.683396 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 22:53:44.683753 (+ 357us) server_negotiation.cc:407] Connection header received
0809 22:53:44.683930 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:53:44.683944 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:53:44.684013 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:53:44.684108 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:53:44.684989 (+ 881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:44.685716 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:44.687008 (+ 1292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:44.687506 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:44.687646 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:44.691658 (+ 4012us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.113.130:56387: BlockingRecv error: recv got EOF from 127.15.113.130:56387 (error 108)
Metrics: {"server-negotiator.queue_time_us":79}
Aug 09 22:53:45 dist-test-slave-ncp9 krb5kdc[29136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.113.131@KRBTEST.COM for oryx/127.15.113.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 22:53:45 dist-test-slave-ncp9 krb5kdc[29136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.113.131@KRBTEST.COM for oryx/127.15.113.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 22:53:45.138761 30700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:45.130920 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:44011 (local address 127.15.113.131:60545)
0809 22:53:45.131384 (+ 464us) negotiation.cc:107] Waiting for socket to connect
0809 22:53:45.131401 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 22:53:45.131509 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:53:45.131769 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:53:45.131772 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:53:45.132161 (+ 389us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:53:45.132318 (+ 157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:53:45.132327 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:53:45.133292 (+ 965us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:53:45.133296 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:53:45.134223 (+ 927us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:53:45.134235 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:53:45.134831 (+ 596us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:53:45.134834 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:53:45.134961 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:45.135538 (+ 577us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:53:45.135561 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:53:45.138520 (+ 2959us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.113.190:44011: Server oryx/127.15.113.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":368,"thread_start_us":262,"threads_started":1}
W20260809 22:53:45.138975 30255 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:53:45.131003 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.113.131:60545 (local address 127.15.113.190:44011)
0809 22:53:45.131227 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0809 22:53:45.131233 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:53:45.131471 (+ 238us) server_negotiation.cc:407] Connection header received
0809 22:53:45.131597 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:53:45.131601 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:53:45.131659 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:53:45.131734 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:53:45.132446 (+ 712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:45.133180 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:45.134325 (+ 1145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:45.134725 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:45.134814 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:45.138745 (+ 3931us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.113.131:60545: BlockingRecv error: recv got EOF from 127.15.113.131:60545 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
Aug 09 22:53:45 dist-test-slave-ncp9 krb5kdc[29136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.113.129@KRBTEST.COM for oryx/127.15.113.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 22:53:45 dist-test-slave-ncp9 krb5kdc[29136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.113.129@KRBTEST.COM for oryx/127.15.113.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 22:53:45.261734 30701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:45.254192 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:44011 (local address 127.15.113.129:40051)
0809 22:53:45.254667 (+ 475us) negotiation.cc:107] Waiting for socket to connect
0809 22:53:45.254682 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 22:53:45.254802 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:53:45.255085 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:53:45.255088 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:53:45.255472 (+ 384us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:53:45.255633 (+ 161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:53:45.255642 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:53:45.256590 (+ 948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:53:45.256593 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:53:45.257635 (+ 1042us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:53:45.257649 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:53:45.258320 (+ 671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:53:45.258323 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:53:45.258431 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:45.258952 (+ 521us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:53:45.258969 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:53:45.261499 (+ 2530us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.113.190:44011: Server oryx/127.15.113.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":369,"thread_start_us":230,"threads_started":1}
W20260809 22:53:45.261894 30255 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:53:45.254253 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.113.129:40051 (local address 127.15.113.190:44011)
0809 22:53:45.254458 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0809 22:53:45.254464 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:53:45.254750 (+ 286us) server_negotiation.cc:407] Connection header received
0809 22:53:45.254903 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:53:45.254906 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:53:45.254984 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:53:45.255056 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:53:45.255755 (+ 699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:45.256488 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:45.257757 (+ 1269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:45.258188 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:45.258277 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:45.261673 (+ 3396us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.113.129:40051: BlockingRecv error: recv got EOF from 127.15.113.129:40051 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
Aug 09 22:53:45 dist-test-slave-ncp9 krb5kdc[29136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.113.130@KRBTEST.COM for oryx/127.15.113.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 22:53:45 dist-test-slave-ncp9 krb5kdc[29136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.113.130@KRBTEST.COM for oryx/127.15.113.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 22:53:45.700434 30702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:45.692777 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:44011 (local address 127.15.113.130:42337)
0809 22:53:45.693320 (+ 543us) negotiation.cc:107] Waiting for socket to connect
0809 22:53:45.693337 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 22:53:45.693450 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:53:45.693705 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:53:45.693709 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:53:45.694077 (+ 368us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:53:45.694245 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:53:45.694255 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:53:45.695186 (+ 931us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:53:45.695189 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:53:45.696116 (+ 927us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:53:45.696128 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:53:45.696779 (+ 651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:53:45.696784 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:53:45.696915 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:45.697493 (+ 578us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:53:45.697510 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:53:45.700129 (+ 2619us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.113.190:44011: Server oryx/127.15.113.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":433,"thread_start_us":282,"threads_started":1}
W20260809 22:53:45.700573 30255 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:53:45.692881 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.113.130:42337 (local address 127.15.113.190:44011)
0809 22:53:45.693065 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0809 22:53:45.693070 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:53:45.693399 (+ 329us) server_negotiation.cc:407] Connection header received
0809 22:53:45.693534 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:53:45.693538 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:53:45.693598 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:53:45.693676 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:53:45.694357 (+ 681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:45.695086 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:45.696241 (+ 1155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:45.696646 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:45.696735 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:45.700327 (+ 3592us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.113.130:42337: BlockingRecv error: recv got EOF from 127.15.113.130:42337 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Aug 09 22:53:46 dist-test-slave-ncp9 krb5kdc[29136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.113.131@KRBTEST.COM for oryx/127.15.113.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 22:53:46 dist-test-slave-ncp9 krb5kdc[29136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.113.131@KRBTEST.COM for oryx/127.15.113.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 22:53:46.147680 30703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:46.139580 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:44011 (local address 127.15.113.131:33373)
0809 22:53:46.140065 (+ 485us) negotiation.cc:107] Waiting for socket to connect
0809 22:53:46.140081 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 22:53:46.140229 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:53:46.140536 (+ 307us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:53:46.140541 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:53:46.140987 (+ 446us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:53:46.141161 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:53:46.141171 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:53:46.142156 (+ 985us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:53:46.142161 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:53:46.143104 (+ 943us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:53:46.143118 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:53:46.143896 (+ 778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:53:46.143904 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:53:46.144047 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:46.144624 (+ 577us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:53:46.144641 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:53:46.147370 (+ 2729us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.113.190:44011: Server oryx/127.15.113.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":390,"thread_start_us":242,"threads_started":1}
W20260809 22:53:46.147804 30255 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:53:46.139638 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.113.131:33373 (local address 127.15.113.190:44011)
0809 22:53:46.139813 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0809 22:53:46.139820 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:53:46.140163 (+ 343us) server_negotiation.cc:407] Connection header received
0809 22:53:46.140344 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:53:46.140349 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:53:46.140410 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:53:46.140491 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:53:46.141297 (+ 806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:46.142018 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:46.143265 (+ 1247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:46.143716 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:46.143821 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:46.147573 (+ 3752us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.113.131:33373: BlockingRecv error: recv got EOF from 127.15.113.131:33373 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
Aug 09 22:53:46 dist-test-slave-ncp9 krb5kdc[29136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.113.129@KRBTEST.COM for oryx/127.15.113.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 22:53:46 dist-test-slave-ncp9 krb5kdc[29136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.113.129@KRBTEST.COM for oryx/127.15.113.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 22:53:46.270414 30704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:46.262533 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:44011 (local address 127.15.113.129:55647)
0809 22:53:46.262963 (+ 430us) negotiation.cc:107] Waiting for socket to connect
0809 22:53:46.262978 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 22:53:46.263093 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:53:46.263370 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:53:46.263373 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:53:46.263758 (+ 385us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:53:46.263936 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:53:46.263946 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:53:46.265204 (+ 1258us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:53:46.265208 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:53:46.266110 (+ 902us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:53:46.266122 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:53:46.266862 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:53:46.266867 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:53:46.266994 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:46.267502 (+ 508us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:53:46.267517 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:53:46.270173 (+ 2656us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.113.190:44011: Server oryx/127.15.113.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":334,"thread_start_us":226,"threads_started":1}
W20260809 22:53:46.270550 30255 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:53:46.262607 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.113.129:55647 (local address 127.15.113.190:44011)
0809 22:53:46.262884 (+ 277us) server_negotiation.cc:206] Beginning negotiation
0809 22:53:46.262890 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:53:46.263043 (+ 153us) server_negotiation.cc:407] Connection header received
0809 22:53:46.263199 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:53:46.263203 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:53:46.263265 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:53:46.263342 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:53:46.264077 (+ 735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:46.265066 (+ 989us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:46.266218 (+ 1152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:46.266745 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:46.266837 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:46.270339 (+ 3502us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.113.129:55647: BlockingRecv error: recv got EOF from 127.15.113.129:55647 (error 108)
Metrics: {"server-negotiator.queue_time_us":151}
Aug 09 22:53:46 dist-test-slave-ncp9 krb5kdc[29136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.113.130@KRBTEST.COM for oryx/127.15.113.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 22:53:46 dist-test-slave-ncp9 krb5kdc[29136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.113.130@KRBTEST.COM for oryx/127.15.113.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 22:53:46.709038 30705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:46.701362 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:44011 (local address 127.15.113.130:49555)
0809 22:53:46.701804 (+ 442us) negotiation.cc:107] Waiting for socket to connect
0809 22:53:46.701820 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 22:53:46.701935 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:53:46.702201 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:53:46.702204 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:53:46.702569 (+ 365us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:53:46.702759 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:53:46.702769 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:53:46.703669 (+ 900us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:53:46.703672 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:53:46.704560 (+ 888us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:53:46.704570 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:53:46.705247 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:53:46.705251 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:53:46.705376 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:46.705908 (+ 532us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:53:46.705925 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:53:46.708764 (+ 2839us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.113.190:44011: Server oryx/127.15.113.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":342,"thread_start_us":221,"threads_started":1}
W20260809 22:53:46.709146 30255 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:53:46.701464 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.113.130:49555 (local address 127.15.113.190:44011)
0809 22:53:46.701685 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0809 22:53:46.701690 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:53:46.701889 (+ 199us) server_negotiation.cc:407] Connection header received
0809 22:53:46.702019 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:53:46.702023 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:53:46.702085 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:53:46.702162 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:53:46.702864 (+ 702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:46.703563 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:46.704666 (+ 1103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:46.705131 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:46.705230 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:46.708968 (+ 3738us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.113.130:49555: BlockingRecv error: recv got EOF from 127.15.113.130:49555 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
Aug 09 22:53:47 dist-test-slave-ncp9 krb5kdc[29136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.113.131@KRBTEST.COM for oryx/127.15.113.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 22:53:47 dist-test-slave-ncp9 krb5kdc[29136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.113.131@KRBTEST.COM for oryx/127.15.113.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 22:53:47.156471 30707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:47.148543 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:44011 (local address 127.15.113.131:44745)
0809 22:53:47.149045 (+ 502us) negotiation.cc:107] Waiting for socket to connect
0809 22:53:47.149060 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 22:53:47.149194 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:53:47.149521 (+ 327us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:53:47.149527 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:53:47.149927 (+ 400us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:53:47.150111 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:53:47.150121 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:53:47.151100 (+ 979us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:53:47.151105 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:53:47.152039 (+ 934us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:53:47.152052 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:53:47.152755 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:53:47.152761 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:53:47.152963 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:47.153507 (+ 544us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:53:47.153523 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:53:47.156193 (+ 2670us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.113.190:44011: Server oryx/127.15.113.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":405,"thread_start_us":252,"threads_started":1}
W20260809 22:53:47.156661 30255 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:53:47.148640 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.113.131:44745 (local address 127.15.113.190:44011)
0809 22:53:47.148839 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0809 22:53:47.148845 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:53:47.149148 (+ 303us) server_negotiation.cc:407] Connection header received
0809 22:53:47.149321 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:53:47.149325 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:53:47.149389 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:53:47.149471 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:53:47.150251 (+ 780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:47.150955 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:47.152182 (+ 1227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:47.152614 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:47.152724 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:47.156430 (+ 3706us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.113.131:44745: BlockingRecv error: recv got EOF from 127.15.113.131:44745 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
Aug 09 22:53:47 dist-test-slave-ncp9 krb5kdc[29136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.113.129@KRBTEST.COM for oryx/127.15.113.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 22:53:47 dist-test-slave-ncp9 krb5kdc[29136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.113.129@KRBTEST.COM for oryx/127.15.113.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 22:53:47.279070 30708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:47.271480 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:44011 (local address 127.15.113.129:46341)
0809 22:53:47.272019 (+ 539us) negotiation.cc:107] Waiting for socket to connect
0809 22:53:47.272034 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 22:53:47.272146 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:53:47.272506 (+ 360us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:53:47.272511 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:53:47.272982 (+ 471us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:53:47.273159 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:53:47.273170 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:53:47.274105 (+ 935us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:53:47.274108 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:53:47.275019 (+ 911us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:53:47.275030 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:53:47.275665 (+ 635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:53:47.275669 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:53:47.275777 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:47.276343 (+ 566us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:53:47.276361 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:53:47.278829 (+ 2468us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.113.190:44011: Server oryx/127.15.113.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":431,"thread_start_us":287,"threads_started":1}
W20260809 22:53:47.279222 30255 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:53:47.271543 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.113.129:46341 (local address 127.15.113.190:44011)
0809 22:53:47.271748 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0809 22:53:47.271753 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:53:47.272113 (+ 360us) server_negotiation.cc:407] Connection header received
0809 22:53:47.272286 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:53:47.272290 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:53:47.272361 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:53:47.272458 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:53:47.273281 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:47.273998 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:47.275132 (+ 1134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:47.275556 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:47.275641 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:47.279007 (+ 3366us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.113.129:46341: BlockingRecv error: recv got EOF from 127.15.113.129:46341 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
Aug 09 22:53:47 dist-test-slave-ncp9 krb5kdc[29136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.113.130@KRBTEST.COM for oryx/127.15.113.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 22:53:47 dist-test-slave-ncp9 krb5kdc[29136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.113.130@KRBTEST.COM for oryx/127.15.113.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 22:53:47.718407 30709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:47.710330 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:44011 (local address 127.15.113.130:47915)
0809 22:53:47.710883 (+ 553us) negotiation.cc:107] Waiting for socket to connect
0809 22:53:47.710898 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 22:53:47.711030 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:53:47.711349 (+ 319us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:53:47.711354 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:53:47.711829 (+ 475us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:53:47.712013 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:53:47.712027 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:53:47.713174 (+ 1147us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:53:47.713179 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:53:47.714160 (+ 981us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:53:47.714175 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:53:47.714825 (+ 650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:53:47.714829 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:53:47.714954 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:47.715481 (+ 527us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:53:47.715498 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:53:47.718160 (+ 2662us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.113.190:44011: Server oryx/127.15.113.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":446,"thread_start_us":292,"threads_started":1}
W20260809 22:53:47.718577 30255 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:53:47.710390 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.113.130:47915 (local address 127.15.113.190:44011)
0809 22:53:47.710574 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0809 22:53:47.710581 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:53:47.710983 (+ 402us) server_negotiation.cc:407] Connection header received
0809 22:53:47.711141 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:53:47.711147 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:53:47.711219 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:53:47.711302 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:53:47.712160 (+ 858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:47.713007 (+ 847us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:47.714281 (+ 1274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:47.714711 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:47.714801 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:47.718367 (+ 3566us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.113.130:47915: BlockingRecv error: recv got EOF from 127.15.113.130:47915 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
I20260809 22:53:48.010136 15814 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskItYPJR/build/asan/bin/kudu pid 30020
I20260809 22:53:48.021158 30712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:48.011028 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35102 (local address 127.15.113.129:39729)
0809 22:53:48.011554 (+ 526us) server_negotiation.cc:206] Beginning negotiation
0809 22:53:48.011563 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 22:53:48.011589 (+ 26us) server_negotiation.cc:407] Connection header received
0809 22:53:48.011647 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:53:48.011655 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:53:48.011876 (+ 221us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:53:48.012072 (+ 196us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:53:48.013394 (+ 1322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:48.014431 (+ 1037us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:48.015547 (+ 1116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:48.016071 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:48.016169 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:48.018440 (+ 2271us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:53:48.018448 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:53:48.018462 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:53:48.018492 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:53:48.020657 (+ 2165us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.15.113.129@KRBTEST.COM not found in keytab (ticket kvno 2)
0809 22:53:48.020926 (+ 269us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:35102: Request ticket server kudu/127.15.113.129@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":368,"thread_start_us":231,"threads_started":1}
W20260809 22:53:48.021250 30712 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:35102: Request ticket server kudu/127.15.113.129@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260809 22:53:48.021289 15814 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskItYPJR/build/asan/bin/kudu pid 30020: Not authorized: Client connection negotiation failed: client connection to 127.15.113.129:39729: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.15.113.129@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260809 22:53:48.021528 15814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskItYPJR/build/asan/bin/kudu with pid 30020
I20260809 22:53:48.046069 15814 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskItYPJR/build/asan/bin/kudu pid 30256
I20260809 22:53:48.057655 30717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:48.047061 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40342 (local address 127.15.113.130:37943)
0809 22:53:48.047651 (+ 590us) server_negotiation.cc:206] Beginning negotiation
0809 22:53:48.047658 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:53:48.047689 (+ 31us) server_negotiation.cc:407] Connection header received
0809 22:53:48.047749 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:53:48.047755 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:53:48.047966 (+ 211us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:53:48.048159 (+ 193us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:53:48.049493 (+ 1334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:48.050553 (+ 1060us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:48.051742 (+ 1189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:48.052299 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:48.052412 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:48.054825 (+ 2413us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:53:48.054833 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:53:48.054846 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:53:48.054878 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:53:48.057016 (+ 2138us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.15.113.130@KRBTEST.COM not found in keytab (ticket kvno 2)
0809 22:53:48.057317 (+ 301us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:40342: Request ticket server kudu/127.15.113.130@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":438,"thread_start_us":275,"threads_started":1}
W20260809 22:53:48.057705 15814 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskItYPJR/build/asan/bin/kudu pid 30256: Not authorized: Client connection negotiation failed: client connection to 127.15.113.130:37943: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.15.113.130@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260809 22:53:48.057753 30717 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:40342: Request ticket server kudu/127.15.113.130@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260809 22:53:48.057830 15814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskItYPJR/build/asan/bin/kudu with pid 30256
I20260809 22:53:48.081343 15814 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskItYPJR/build/asan/bin/kudu pid 30446
I20260809 22:53:48.092715 30718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:48.082296 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47288 (local address 127.15.113.131:38311)
0809 22:53:48.082841 (+ 545us) server_negotiation.cc:206] Beginning negotiation
0809 22:53:48.082848 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:53:48.082872 (+ 24us) server_negotiation.cc:407] Connection header received
0809 22:53:48.082937 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:53:48.082943 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:53:48.083146 (+ 203us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:53:48.083326 (+ 180us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:53:48.084640 (+ 1314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:48.085741 (+ 1101us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:48.086920 (+ 1179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:48.087466 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:48.087582 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:48.090004 (+ 2422us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:53:48.090012 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:53:48.090025 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:53:48.090055 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:53:48.092112 (+ 2057us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.15.113.131@KRBTEST.COM not found in keytab (ticket kvno 2)
0809 22:53:48.092394 (+ 282us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:47288: Request ticket server kudu/127.15.113.131@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":411,"thread_start_us":268,"threads_started":1}
W20260809 22:53:48.092746 15814 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskItYPJR/build/asan/bin/kudu pid 30446: Not authorized: Client connection negotiation failed: client connection to 127.15.113.131:38311: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.15.113.131@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260809 22:53:48.092818 30718 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:47288: Request ticket server kudu/127.15.113.131@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260809 22:53:48.092918 15814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskItYPJR/build/asan/bin/kudu with pid 30446
I20260809 22:53:48.113344 15814 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskItYPJR/build/asan/bin/kudu pid 29924
I20260809 22:53:48.114550 30010 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:53:48.230001 15814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskItYPJR/build/asan/bin/kudu with pid 29924
2026-08-09T22:53:48Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (21132 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-08-09T22:53:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T22:53:48Z Disabled control of system clock
I20260809 22:53:48.316252 15814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskItYPJR/build/asan/bin/kudu
/tmp/dist-test-taskItYPJR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786315935617919-15814-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786315935617919-15814-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786315935617919-15814-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786315935617919-15814-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.113.190:33523
--webserver_interface=127.15.113.190
--webserver_port=0
--builtin_ntp_servers=127.15.113.148:46053
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.113.190:33523
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260809 22:53:48.637933 30725 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:53:48.638213 30725 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:53:48.638299 30725 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:53:48.645273 30725 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 22:53:48.645364 30725 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:53:48.645408 30725 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:53:48.645462 30725 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 22:53:48.645506 30725 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 22:53:48.654148 30725 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.113.148:46053
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786315935617919-15814-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786315935617919-15814-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.113.190:33523
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.113.190:33523
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786315935617919-15814-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.113.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786315935617919-15814-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:53:48.655316 30725 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:53:48.657042 30725 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:53:48.668232 30738 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:53:48.668313 30734 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:53:48.668275 30735 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:53:48.668951 30725 server_base.cc:1034] running on GCE node
I20260809 22:53:48.669700 30725 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:53:48.670998 30725 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:53:48.672199 30725 hybrid_clock.cc:648] HybridClock initialized: now 1786316028672151 us; error 40 us; skew 500 ppm
I20260809 22:53:48.672518 30725 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 22:53:48.674672 30725 webserver.cc:460] Webserver started at http://127.15.113.190:43363/ using document root <none> and password file <none>
I20260809 22:53:48.675318 30725 fs_manager.cc:359] Metadata directory not provided
I20260809 22:53:48.675421 30725 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:53:48.675735 30725 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:53:48.678529 30725 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786315935617919-15814-0/minicluster-data/master-0/data/instance:
uuid: "d22c4086fdf14229827d83c13162bab0"
format_stamp: "Formatted at 2026-08-09 22:53:48 on dist-test-slave-ncp9"
server_key: "86ec704b699d86ea900beb13d1e71728"
server_key_iv: "7f9a25480f3cb83bca18120cfd371eb4"
server_key_version: "encryptionkey@0"
I20260809 22:53:48.679278 30725 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786315935617919-15814-0/minicluster-data/master-0/wal/instance:
uuid: "d22c4086fdf14229827d83c13162bab0"
format_stamp: "Formatted at 2026-08-09 22:53:48 on dist-test-slave-ncp9"
server_key: "86ec704b699d86ea900beb13d1e71728"
server_key_iv: "7f9a25480f3cb83bca18120cfd371eb4"
server_key_version: "encryptionkey@0"
I20260809 22:53:48.684166 30725 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260809 22:53:48.687453 30750 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:53:48.688663 30725 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 22:53:48.688823 30725 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786315935617919-15814-0/minicluster-data/master-0/data,/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786315935617919-15814-0/minicluster-data/master-0/wal
uuid: "d22c4086fdf14229827d83c13162bab0"
format_stamp: "Formatted at 2026-08-09 22:53:48 on dist-test-slave-ncp9"
server_key: "86ec704b699d86ea900beb13d1e71728"
server_key_iv: "7f9a25480f3cb83bca18120cfd371eb4"
server_key_version: "encryptionkey@0"
I20260809 22:53:48.689056 30725 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786315935617919-15814-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786315935617919-15814-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786315935617919-15814-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:53:48.698884 30725 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:53:48.699617 30725 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:53:48.699896 30725 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:53:48.716595 30725 rpc_server.cc:307] RPC server started. Bound to: 127.15.113.190:33523
I20260809 22:53:48.716640 30831 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.113.190:33523 every 8 connection(s)
I20260809 22:53:48.717967 30725 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786315935617919-15814-0/minicluster-data/master-0/data/info.pb
I20260809 22:53:48.721455 30833 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:53:48.726985 15814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskItYPJR/build/asan/bin/kudu as pid 30725
I20260809 22:53:48.727147 15814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786315935617919-15814-0/minicluster-data/master-0/wal/instance
I20260809 22:53:48.727797 15814 external_mini_cluster.cc:1428] Setting key acc65a6143b7acc0ba21c139fbcd3d02
I20260809 22:53:48.731956 30833 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d22c4086fdf14229827d83c13162bab0: Bootstrap starting.
I20260809 22:53:48.735747 30833 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d22c4086fdf14229827d83c13162bab0: Neither blocks nor log segments found. Creating new log.
I20260809 22:53:48.736886 30833 log.cc:824] T 00000000000000000000000000000000 P d22c4086fdf14229827d83c13162bab0: Log is configured to *not* fsync() on all Append() calls
I20260809 22:53:48.739984 30833 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d22c4086fdf14229827d83c13162bab0: No bootstrap required, opened a new log
I20260809 22:53:48.742833 30838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:48.729755 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39030 (local address 127.15.113.190:33523)
0809 22:53:48.730802 (+ 1047us) server_negotiation.cc:206] Beginning negotiation
0809 22:53:48.730815 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 22:53:48.730865 (+ 50us) server_negotiation.cc:407] Connection header received
0809 22:53:48.733061 (+ 2196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:53:48.733088 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:53:48.733661 (+ 573us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:53:48.734072 (+ 411us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:53:48.734575 (+ 503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:48.736449 (+ 1874us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:48.738011 (+ 1562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:48.738920 (+ 909us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:48.739099 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:48.740134 (+ 1035us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:53:48.740145 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:53:48.740163 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 22:53:48.740851 (+ 688us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:53:48.740984 (+ 133us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 22:53:48.741010 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:53:48.741135 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0809 22:53:48.741305 (+ 170us) server_negotiation.cc:299] Negotiation successful
0809 22:53:48.741598 (+ 293us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":531,"thread_start_us":317,"threads_started":1}
I20260809 22:53:48.748904 30833 raft_consensus.cc:348] T 00000000000000000000000000000000 P d22c4086fdf14229827d83c13162bab0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d22c4086fdf14229827d83c13162bab0" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 33523 } }
I20260809 22:53:48.749277 30833 raft_consensus.cc:374] T 00000000000000000000000000000000 P d22c4086fdf14229827d83c13162bab0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:53:48.749353 30833 raft_consensus.cc:729] T 00000000000000000000000000000000 P d22c4086fdf14229827d83c13162bab0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22c4086fdf14229827d83c13162bab0, State: Initialized, Role: FOLLOWER
I20260809 22:53:48.749842 30833 consensus_queue.cc:260] T 00000000000000000000000000000000 P d22c4086fdf14229827d83c13162bab0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d22c4086fdf14229827d83c13162bab0" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 33523 } }
I20260809 22:53:48.750031 30833 raft_consensus.cc:388] T 00000000000000000000000000000000 P d22c4086fdf14229827d83c13162bab0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 22:53:48.750128 30833 raft_consensus.cc:482] T 00000000000000000000000000000000 P d22c4086fdf14229827d83c13162bab0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 22:53:48.750259 30833 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d22c4086fdf14229827d83c13162bab0 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:53:48.752226 30833 raft_consensus.cc:504] T 00000000000000000000000000000000 P d22c4086fdf14229827d83c13162bab0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d22c4086fdf14229827d83c13162bab0" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 33523 } }
I20260809 22:53:48.752638 30833 leader_election.cc:302] T 00000000000000000000000000000000 P d22c4086fdf14229827d83c13162bab0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d22c4086fdf14229827d83c13162bab0; no voters:
I20260809 22:53:48.753185 30833 leader_election.cc:288] T 00000000000000000000000000000000 P d22c4086fdf14229827d83c13162bab0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 22:53:48.753299 30840 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d22c4086fdf14229827d83c13162bab0 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 22:53:48.753935 30840 raft_consensus.cc:686] T 00000000000000000000000000000000 P d22c4086fdf14229827d83c13162bab0 [term 1 LEADER]: Becoming Leader. State: Replica: d22c4086fdf14229827d83c13162bab0, State: Running, Role: LEADER
I20260809 22:53:48.754474 30840 consensus_queue.cc:237] T 00000000000000000000000000000000 P d22c4086fdf14229827d83c13162bab0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d22c4086fdf14229827d83c13162bab0" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 33523 } }
I20260809 22:53:48.754926 30833 sys_catalog.cc:564] T 00000000000000000000000000000000 P d22c4086fdf14229827d83c13162bab0 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 22:53:48.757210 30842 sys_catalog.cc:455] T 00000000000000000000000000000000 P d22c4086fdf14229827d83c13162bab0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d22c4086fdf14229827d83c13162bab0. Latest consensus state: current_term: 1 leader_uuid: "d22c4086fdf14229827d83c13162bab0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d22c4086fdf14229827d83c13162bab0" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 33523 } } }
I20260809 22:53:48.757822 30842 sys_catalog.cc:458] T 00000000000000000000000000000000 P d22c4086fdf14229827d83c13162bab0 [sys.catalog]: This master's current role is: LEADER
I20260809 22:53:48.757413 30841 sys_catalog.cc:455] T 00000000000000000000000000000000 P d22c4086fdf14229827d83c13162bab0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d22c4086fdf14229827d83c13162bab0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d22c4086fdf14229827d83c13162bab0" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 33523 } } }
I20260809 22:53:48.758751 30841 sys_catalog.cc:458] T 00000000000000000000000000000000 P d22c4086fdf14229827d83c13162bab0 [sys.catalog]: This master's current role is: LEADER
I20260809 22:53:48.760020 30849 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 22:53:48.765407 30849 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 22:53:48.773599 30849 catalog_manager.cc:1347] Generated new cluster ID: d704df4905364d60bb0493771dba4ca3
I20260809 22:53:48.773710 30849 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 22:53:48.783812 30849 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 22:53:48.785496 30849 catalog_manager.cc:1504] Loading token signing keys...
I20260809 22:53:48.798745 30849 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d22c4086fdf14229827d83c13162bab0: Generated new TSK 0
I20260809 22:53:48.799748 30849 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 22:53:48.812274 15814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskItYPJR/build/asan/bin/kudu
/tmp/dist-test-taskItYPJR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786315935617919-15814-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786315935617919-15814-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786315935617919-15814-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786315935617919-15814-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.113.129:0
--local_ip_for_outbound_sockets=127.15.113.129
--webserver_interface=127.15.113.129
--webserver_port=0
--tserver_master_addrs=127.15.113.190:33523
--builtin_ntp_servers=127.15.113.148:46053
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260809 22:53:49.159385 30869 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:53:49.159644 30869 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:53:49.159734 30869 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:53:49.166742 30869 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:53:49.166829 30869 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:53:49.166942 30869 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.113.129
I20260809 22:53:49.175573 30869 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.113.148:46053
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786315935617919-15814-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786315935617919-15814-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.113.129:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786315935617919-15814-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.113.129
--webserver_port=0
--tserver_master_addrs=127.15.113.190:33523
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.113.129
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786315935617919-15814-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:53:49.176689 30869 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:53:49.178422 30869 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:53:49.191066 30879 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:53:49.191133 30877 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:53:49.191107 30876 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:53:49.192094 30869 server_base.cc:1034] running on GCE node
I20260809 22:53:49.192698 30869 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:53:49.193688 30869 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:53:49.194887 30869 hybrid_clock.cc:648] HybridClock initialized: now 1786316029194840 us; error 35 us; skew 500 ppm
I20260809 22:53:49.195209 30869 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 22:53:49.199784 30869 webserver.cc:460] Webserver started at http://127.15.113.129:36323/ using document root <none> and password file <none>
I20260809 22:53:49.200470 30869 fs_manager.cc:359] Metadata directory not provided
I20260809 22:53:49.200589 30869 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:53:49.200963 30869 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:53:49.203820 30869 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786315935617919-15814-0/minicluster-data/ts-0/data/instance:
uuid: "93136c194e6a4263928c371d01fda6a8"
format_stamp: "Formatted at 2026-08-09 22:53:49 on dist-test-slave-ncp9"
server_key: "8516333cc1edcdd27222784bdeac7e45"
server_key_iv: "001894489fa0fb82267f916348842022"
server_key_version: "encryptionkey@0"
I20260809 22:53:49.204633 30869 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786315935617919-15814-0/minicluster-data/ts-0/wal/instance:
uuid: "93136c194e6a4263928c371d01fda6a8"
format_stamp: "Formatted at 2026-08-09 22:53:49 on dist-test-slave-ncp9"
server_key: "8516333cc1edcdd27222784bdeac7e45"
server_key_iv: "001894489fa0fb82267f916348842022"
server_key_version: "encryptionkey@0"
I20260809 22:53:49.209723 30869 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.001s
I20260809 22:53:49.213105 30888 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:53:49.214406 30869 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 22:53:49.214551 30869 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786315935617919-15814-0/minicluster-data/ts-0/data,/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786315935617919-15814-0/minicluster-data/ts-0/wal
uuid: "93136c194e6a4263928c371d01fda6a8"
format_stamp: "Formatted at 2026-08-09 22:53:49 on dist-test-slave-ncp9"
server_key: "8516333cc1edcdd27222784bdeac7e45"
server_key_iv: "001894489fa0fb82267f916348842022"
server_key_version: "encryptionkey@0"
I20260809 22:53:49.214751 30869 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786315935617919-15814-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786315935617919-15814-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786315935617919-15814-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:53:49.234289 30869 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:53:49.235037 30869 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:53:49.235316 30869 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:53:49.236240 30869 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:53:49.237645 30869 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:53:49.237739 30869 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:53:49.237840 30869 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:53:49.237915 30869 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:53:49.267894 30869 rpc_server.cc:307] RPC server started. Bound to: 127.15.113.129:43495
I20260809 22:53:49.267956 31051 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.113.129:43495 every 8 connection(s)
I20260809 22:53:49.269205 30869 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786315935617919-15814-0/minicluster-data/ts-0/data/info.pb
I20260809 22:53:49.274317 15814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskItYPJR/build/asan/bin/kudu as pid 30869
I20260809 22:53:49.274485 15814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786315935617919-15814-0/minicluster-data/ts-0/wal/instance
I20260809 22:53:49.275105 15814 external_mini_cluster.cc:1428] Setting key af3c1916ebc7e7f858085261f486546f
I20260809 22:53:49.277738 15814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskItYPJR/build/asan/bin/kudu
/tmp/dist-test-taskItYPJR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786315935617919-15814-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786315935617919-15814-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786315935617919-15814-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786315935617919-15814-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.113.130:0
--local_ip_for_outbound_sockets=127.15.113.130
--webserver_interface=127.15.113.130
--webserver_port=0
--tserver_master_addrs=127.15.113.190:33523
--builtin_ntp_servers=127.15.113.148:46053
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260809 22:53:49.281383 31054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:49.271556 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.113.129:33519 (local address 127.15.113.190:33523)
0809 22:53:49.271950 (+ 394us) server_negotiation.cc:206] Beginning negotiation
0809 22:53:49.271954 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 22:53:49.273113 (+ 1159us) server_negotiation.cc:407] Connection header received
0809 22:53:49.274898 (+ 1785us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:53:49.274903 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:53:49.274981 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:53:49.275077 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:53:49.276266 (+ 1189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:49.277176 (+ 910us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:49.278778 (+ 1602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:49.279208 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:49.279310 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:49.280573 (+ 1263us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:53:49.280577 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:53:49.280580 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 22:53:49.280597 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:53:49.280637 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 22:53:49.280656 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:53:49.280728 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0809 22:53:49.280955 (+ 227us) server_negotiation.cc:299] Negotiation successful
0809 22:53:49.281045 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":299,"thread_start_us":208,"threads_started":1}
I20260809 22:53:49.282308 31058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:49.272051 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:33523 (local address 127.15.113.129:33519)
0809 22:53:49.272973 (+ 922us) negotiation.cc:107] Waiting for socket to connect
0809 22:53:49.273017 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 22:53:49.274651 (+ 1634us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:53:49.275339 (+ 688us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:53:49.275346 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:53:49.275427 (+ 81us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:53:49.276092 (+ 665us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:53:49.276112 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:53:49.277319 (+ 1207us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:53:49.277323 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:53:49.278642 (+ 1319us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:53:49.278656 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:53:49.279423 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:53:49.279431 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:53:49.279638 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:49.280290 (+ 652us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 22:53:49.280293 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:53:49.280329 (+ 36us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 22:53:49.280405 (+ 76us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:53:49.280742 (+ 337us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:53:49.280753 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:53:49.280761 (+ 8us) client_negotiation.cc:769] Sending connection context
0809 22:53:49.280926 (+ 165us) client_negotiation.cc:240] Negotiation successful
0809 22:53:49.281306 (+ 380us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":473,"thread_start_us":276,"threads_started":1}
I20260809 22:53:49.283679 31052 heartbeater.cc:347] Connected to a master server at 127.15.113.190:33523
I20260809 22:53:49.283933 31052 heartbeater.cc:464] Registering TS with master...
I20260809 22:53:49.284613 31052 heartbeater.cc:511] Master 127.15.113.190:33523 requested a full tablet report, sending...
I20260809 22:53:49.286731 30779 ts_manager.cc:194] Registered new tserver with Master: 93136c194e6a4263928c371d01fda6a8 (127.15.113.129:43495)
I20260809 22:53:49.288570 30779 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.15.113.129:33519
W20260809 22:53:49.611263 31059 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:53:49.611516 31059 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:53:49.611595 31059 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:53:49.618491 31059 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:53:49.618610 31059 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:53:49.618746 31059 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.113.130
I20260809 22:53:49.627487 31059 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.113.148:46053
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786315935617919-15814-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786315935617919-15814-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.113.130:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786315935617919-15814-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.113.130
--webserver_port=0
--tserver_master_addrs=127.15.113.190:33523
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.113.130
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786315935617919-15814-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:53:49.628696 31059 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:53:49.630434 31059 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:53:49.642886 31064 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:53:49.643045 31070 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:53:49.643074 31075 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:53:49.643785 31059 server_base.cc:1034] running on GCE node
I20260809 22:53:49.644381 31059 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:53:49.645331 31059 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:53:49.646529 31059 hybrid_clock.cc:648] HybridClock initialized: now 1786316029646478 us; error 43 us; skew 500 ppm
I20260809 22:53:49.646852 31059 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 22:53:49.649061 31059 webserver.cc:460] Webserver started at http://127.15.113.130:36793/ using document root <none> and password file <none>
I20260809 22:53:49.649776 31059 fs_manager.cc:359] Metadata directory not provided
I20260809 22:53:49.649885 31059 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:53:49.650208 31059 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:53:49.653059 31059 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786315935617919-15814-0/minicluster-data/ts-1/data/instance:
uuid: "6804a3985e6a42baab96d3d6faf2b4c9"
format_stamp: "Formatted at 2026-08-09 22:53:49 on dist-test-slave-ncp9"
server_key: "c8bcef864a0a77f1b8275138aedaeab2"
server_key_iv: "362fef36a7b8427904303406c9011773"
server_key_version: "encryptionkey@0"
I20260809 22:53:49.653848 31059 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786315935617919-15814-0/minicluster-data/ts-1/wal/instance:
uuid: "6804a3985e6a42baab96d3d6faf2b4c9"
format_stamp: "Formatted at 2026-08-09 22:53:49 on dist-test-slave-ncp9"
server_key: "c8bcef864a0a77f1b8275138aedaeab2"
server_key_iv: "362fef36a7b8427904303406c9011773"
server_key_version: "encryptionkey@0"
I20260809 22:53:49.659296 31059 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 22:53:49.662683 31081 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:53:49.663882 31059 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.001s
I20260809 22:53:49.664028 31059 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786315935617919-15814-0/minicluster-data/ts-1/data,/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786315935617919-15814-0/minicluster-data/ts-1/wal
uuid: "6804a3985e6a42baab96d3d6faf2b4c9"
format_stamp: "Formatted at 2026-08-09 22:53:49 on dist-test-slave-ncp9"
server_key: "c8bcef864a0a77f1b8275138aedaeab2"
server_key_iv: "362fef36a7b8427904303406c9011773"
server_key_version: "encryptionkey@0"
I20260809 22:53:49.664227 31059 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786315935617919-15814-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786315935617919-15814-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786315935617919-15814-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:53:49.681449 31059 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:53:49.682212 31059 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:53:49.682482 31059 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:53:49.683328 31059 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:53:49.684664 31059 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:53:49.684769 31059 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:53:49.684912 31059 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:53:49.685001 31059 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.001s
I20260809 22:53:49.715467 31059 rpc_server.cc:307] RPC server started. Bound to: 127.15.113.130:38181
I20260809 22:53:49.715515 31268 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.113.130:38181 every 8 connection(s)
I20260809 22:53:49.716782 31059 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786315935617919-15814-0/minicluster-data/ts-1/data/info.pb
I20260809 22:53:49.717217 15814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskItYPJR/build/asan/bin/kudu as pid 31059
I20260809 22:53:49.717396 15814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786315935617919-15814-0/minicluster-data/ts-1/wal/instance
I20260809 22:53:49.717967 15814 external_mini_cluster.cc:1428] Setting key e296c5ac60205ddb920d7b1284f0c098
I20260809 22:53:49.721464 15814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskItYPJR/build/asan/bin/kudu
/tmp/dist-test-taskItYPJR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786315935617919-15814-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786315935617919-15814-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786315935617919-15814-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786315935617919-15814-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.113.131:0
--local_ip_for_outbound_sockets=127.15.113.131
--webserver_interface=127.15.113.131
--webserver_port=0
--tserver_master_addrs=127.15.113.190:33523
--builtin_ntp_servers=127.15.113.148:46053
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260809 22:53:49.746573 31054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:49.720871 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.113.130:50629 (local address 127.15.113.190:33523)
0809 22:53:49.721076 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0809 22:53:49.721083 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:53:49.737838 (+ 16755us) server_negotiation.cc:407] Connection header received
0809 22:53:49.740170 (+ 2332us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:53:49.740174 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:53:49.740238 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:53:49.740337 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:53:49.741664 (+ 1327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:49.742426 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:49.743968 (+ 1542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:49.744345 (+ 377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:49.744454 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:49.745816 (+ 1362us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:53:49.745821 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:53:49.745824 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 22:53:49.745841 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:53:49.745872 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 22:53:49.745883 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:53:49.745957 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0809 22:53:49.746186 (+ 229us) server_negotiation.cc:299] Negotiation successful
0809 22:53:49.746283 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260809 22:53:49.747536 31272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:49.720328 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:33523 (local address 127.15.113.130:50629)
0809 22:53:49.737679 (+ 17351us) negotiation.cc:107] Waiting for socket to connect
0809 22:53:49.737723 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 22:53:49.739848 (+ 2125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:53:49.740642 (+ 794us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:53:49.740653 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:53:49.740780 (+ 127us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:53:49.741491 (+ 711us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:53:49.741510 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:53:49.742582 (+ 1072us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:53:49.742586 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:53:49.743820 (+ 1234us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:53:49.743839 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:53:49.744565 (+ 726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:53:49.744573 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:53:49.744801 (+ 228us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:49.745522 (+ 721us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 22:53:49.745525 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:53:49.745563 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 22:53:49.745645 (+ 82us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:53:49.745970 (+ 325us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:53:49.745983 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:53:49.745989 (+ 6us) client_negotiation.cc:769] Sending connection context
0809 22:53:49.746135 (+ 146us) client_negotiation.cc:240] Negotiation successful
0809 22:53:49.746518 (+ 383us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16773,"thread_start_us":20029,"threads_started":1}
I20260809 22:53:49.748997 31269 heartbeater.cc:347] Connected to a master server at 127.15.113.190:33523
I20260809 22:53:49.749287 31269 heartbeater.cc:464] Registering TS with master...
I20260809 22:53:49.750011 31269 heartbeater.cc:511] Master 127.15.113.190:33523 requested a full tablet report, sending...
I20260809 22:53:49.751362 30779 ts_manager.cc:194] Registered new tserver with Master: 6804a3985e6a42baab96d3d6faf2b4c9 (127.15.113.130:38181)
I20260809 22:53:49.752148 30779 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.15.113.130:50629
W20260809 22:53:50.050434 31273 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:53:50.050725 31273 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:53:50.050853 31273 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:53:50.057881 31273 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:53:50.057981 31273 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:53:50.058115 31273 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.113.131
I20260809 22:53:50.066993 31273 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.113.148:46053
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786315935617919-15814-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786315935617919-15814-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.113.131:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786315935617919-15814-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.113.131
--webserver_port=0
--tserver_master_addrs=127.15.113.190:33523
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.113.131
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786315935617919-15814-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:53:50.068280 31273 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:53:50.070057 31273 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:53:50.082276 31285 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:53:50.082382 31282 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:53:50.082358 31283 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:53:50.083350 31273 server_base.cc:1034] running on GCE node
I20260809 22:53:50.083940 31273 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:53:50.084811 31273 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:53:50.086004 31273 hybrid_clock.cc:648] HybridClock initialized: now 1786316030085951 us; error 47 us; skew 500 ppm
I20260809 22:53:50.086295 31273 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 22:53:50.088358 31273 webserver.cc:460] Webserver started at http://127.15.113.131:44765/ using document root <none> and password file <none>
I20260809 22:53:50.089059 31273 fs_manager.cc:359] Metadata directory not provided
I20260809 22:53:50.089161 31273 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:53:50.089460 31273 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:53:50.092231 31273 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786315935617919-15814-0/minicluster-data/ts-2/data/instance:
uuid: "77ef0e0517264859a2aebd96b8dca580"
format_stamp: "Formatted at 2026-08-09 22:53:50 on dist-test-slave-ncp9"
server_key: "5390e0815904fb52f8de521fb0ede31f"
server_key_iv: "e9c82ee8b0929b48c2dbfe60c498482e"
server_key_version: "encryptionkey@0"
I20260809 22:53:50.093065 31273 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786315935617919-15814-0/minicluster-data/ts-2/wal/instance:
uuid: "77ef0e0517264859a2aebd96b8dca580"
format_stamp: "Formatted at 2026-08-09 22:53:50 on dist-test-slave-ncp9"
server_key: "5390e0815904fb52f8de521fb0ede31f"
server_key_iv: "e9c82ee8b0929b48c2dbfe60c498482e"
server_key_version: "encryptionkey@0"
I20260809 22:53:50.097805 31273 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260809 22:53:50.101104 31292 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:53:50.102352 31273 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 22:53:50.102499 31273 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786315935617919-15814-0/minicluster-data/ts-2/data,/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786315935617919-15814-0/minicluster-data/ts-2/wal
uuid: "77ef0e0517264859a2aebd96b8dca580"
format_stamp: "Formatted at 2026-08-09 22:53:50 on dist-test-slave-ncp9"
server_key: "5390e0815904fb52f8de521fb0ede31f"
server_key_iv: "e9c82ee8b0929b48c2dbfe60c498482e"
server_key_version: "encryptionkey@0"
I20260809 22:53:50.102689 31273 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786315935617919-15814-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786315935617919-15814-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786315935617919-15814-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:53:50.117208 31273 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:53:50.117949 31273 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:53:50.118219 31273 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:53:50.119134 31273 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:53:50.120464 31273 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:53:50.120560 31273 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:53:50.120675 31273 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:53:50.120750 31273 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:53:50.149915 31273 rpc_server.cc:307] RPC server started. Bound to: 127.15.113.131:43645
I20260809 22:53:50.149964 31462 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.113.131:43645 every 8 connection(s)
I20260809 22:53:50.151118 31273 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786315935617919-15814-0/minicluster-data/ts-2/data/info.pb
I20260809 22:53:50.152822 15814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskItYPJR/build/asan/bin/kudu as pid 31273
I20260809 22:53:50.153009 15814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786315935617919-15814-0/minicluster-data/ts-2/wal/instance
I20260809 22:53:50.153599 15814 external_mini_cluster.cc:1428] Setting key 79bacaab732ed178d2f478359ac7c935
I20260809 22:53:50.163288 31054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:50.153744 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.113.131:45847 (local address 127.15.113.190:33523)
0809 22:53:50.153928 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0809 22:53:50.153932 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 22:53:50.155586 (+ 1654us) server_negotiation.cc:407] Connection header received
0809 22:53:50.157461 (+ 1875us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:53:50.157466 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:53:50.157530 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:53:50.157612 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:53:50.158520 (+ 908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:50.159246 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:50.160867 (+ 1621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:50.161406 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:50.161499 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:50.162685 (+ 1186us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:53:50.162689 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:53:50.162693 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 22:53:50.162710 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:53:50.162742 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 22:53:50.162754 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:53:50.162821 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0809 22:53:50.162970 (+ 149us) server_negotiation.cc:299] Negotiation successful
0809 22:53:50.163046 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260809 22:53:50.164170 31467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:50.154405 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:33523 (local address 127.15.113.131:45847)
0809 22:53:50.155417 (+ 1012us) negotiation.cc:107] Waiting for socket to connect
0809 22:53:50.155464 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0809 22:53:50.157229 (+ 1765us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:53:50.157778 (+ 549us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:53:50.157786 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:53:50.157866 (+ 80us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:53:50.158403 (+ 537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:53:50.158420 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:53:50.159384 (+ 964us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:53:50.159389 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:53:50.160749 (+ 1360us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:53:50.160762 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:53:50.161580 (+ 818us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:53:50.161585 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:53:50.161785 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:50.162405 (+ 620us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 22:53:50.162408 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:53:50.162444 (+ 36us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 22:53:50.162514 (+ 70us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:53:50.162824 (+ 310us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:53:50.162830 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:53:50.162835 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 22:53:50.162953 (+ 118us) client_negotiation.cc:240] Negotiation successful
0809 22:53:50.163216 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":566,"thread_start_us":327,"threads_started":1}
I20260809 22:53:50.165405 31463 heartbeater.cc:347] Connected to a master server at 127.15.113.190:33523
I20260809 22:53:50.165642 31463 heartbeater.cc:464] Registering TS with master...
I20260809 22:53:50.166322 31463 heartbeater.cc:511] Master 127.15.113.190:33523 requested a full tablet report, sending...
I20260809 22:53:50.167654 30779 ts_manager.cc:194] Registered new tserver with Master: 77ef0e0517264859a2aebd96b8dca580 (127.15.113.131:43645)
I20260809 22:53:50.168637 30779 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.15.113.131:45847
I20260809 22:53:50.168879 15814 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260809 22:53:50.174749 31054 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:53:50.173109 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39034 (local address 127.15.113.190:33523)
0809 22:53:50.173254 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0809 22:53:50.173257 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 22:53:50.173765 (+ 508us) server_negotiation.cc:407] Connection header received
0809 22:53:50.173912 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:53:50.173917 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:53:50.173991 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:53:50.174071 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:53:50.174517 (+ 446us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39034: BlockingRecv error: recv got EOF from 127.0.0.1:39034 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
I20260809 22:53:50.175374 15814 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskItYPJR/build/asan/bin/kudu pid 30869
I20260809 22:53:50.182400 31478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:50.176268 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36410 (local address 127.15.113.129:43495)
0809 22:53:50.176767 (+ 499us) server_negotiation.cc:206] Beginning negotiation
0809 22:53:50.176773 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:53:50.176798 (+ 25us) server_negotiation.cc:407] Connection header received
0809 22:53:50.176852 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:53:50.176857 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:53:50.177088 (+ 231us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:53:50.177209 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:53:50.177533 (+ 324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:50.178527 (+ 994us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:50.179716 (+ 1189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:50.180231 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:50.180324 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:50.181168 (+ 844us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:53:50.181175 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:53:50.181186 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 22:53:50.181703 (+ 517us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:53:50.181746 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 22:53:50.181766 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:53:50.181850 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0809 22:53:50.181925 (+ 75us) server_negotiation.cc:299] Negotiation successful
0809 22:53:50.182041 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":350,"thread_start_us":250,"threads_started":1}
I20260809 22:53:50.183528 31044 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:53:50.262635 15814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskItYPJR/build/asan/bin/kudu with pid 30869
I20260809 22:53:50.281731 15814 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskItYPJR/build/asan/bin/kudu pid 31059
I20260809 22:53:50.289371 31480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:50.282647 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45916 (local address 127.15.113.130:38181)
0809 22:53:50.283171 (+ 524us) server_negotiation.cc:206] Beginning negotiation
0809 22:53:50.283178 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:53:50.283204 (+ 26us) server_negotiation.cc:407] Connection header received
0809 22:53:50.283265 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:53:50.283271 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:53:50.283481 (+ 210us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:53:50.283627 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:53:50.284041 (+ 414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:50.285070 (+ 1029us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:50.286265 (+ 1195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:50.286828 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:50.286939 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:50.287900 (+ 961us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:53:50.287908 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:53:50.287921 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 22:53:50.288512 (+ 591us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:53:50.288560 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 22:53:50.288581 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:53:50.288680 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0809 22:53:50.288840 (+ 160us) server_negotiation.cc:299] Negotiation successful
0809 22:53:50.289043 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":373,"thread_start_us":256,"threads_started":1}
I20260809 22:53:50.290638 31263 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:53:50.369886 15814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskItYPJR/build/asan/bin/kudu with pid 31059
I20260809 22:53:50.389238 15814 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskItYPJR/build/asan/bin/kudu pid 31273
I20260809 22:53:50.397509 31482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:50.390196 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54284 (local address 127.15.113.131:43645)
0809 22:53:50.390807 (+ 611us) server_negotiation.cc:206] Beginning negotiation
0809 22:53:50.390814 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:53:50.390845 (+ 31us) server_negotiation.cc:407] Connection header received
0809 22:53:50.390906 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:53:50.390913 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:53:50.391147 (+ 234us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:53:50.391270 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:53:50.391730 (+ 460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:50.392850 (+ 1120us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:50.394244 (+ 1394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:50.394806 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:50.394920 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:50.395962 (+ 1042us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:53:50.395970 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:53:50.395985 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 22:53:50.396567 (+ 582us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:53:50.396615 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 22:53:50.396637 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:53:50.396747 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0809 22:53:50.397030 (+ 283us) server_negotiation.cc:299] Negotiation successful
0809 22:53:50.397198 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":451,"thread_start_us":290,"threads_started":1}
I20260809 22:53:50.398798 31457 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:53:50.478451 15814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskItYPJR/build/asan/bin/kudu with pid 31273
I20260809 22:53:50.499822 15814 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskItYPJR/build/asan/bin/kudu pid 30725
I20260809 22:53:50.501050 30824 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:53:50.601085 15814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskItYPJR/build/asan/bin/kudu with pid 30725
2026-08-09T22:53:50Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (2365 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-08-09T22:53:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T22:53:50Z Disabled control of system clock
I20260809 22:53:50.684656 15814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskItYPJR/build/asan/bin/kudu
/tmp/dist-test-taskItYPJR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786315935617919-15814-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786315935617919-15814-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786315935617919-15814-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786315935617919-15814-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.113.190:33143
--webserver_interface=127.15.113.190
--webserver_port=0
--builtin_ntp_servers=127.15.113.148:40873
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.113.190:33143
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260809 22:53:51.011034 31489 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:53:51.011315 31489 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:53:51.011405 31489 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:53:51.018690 31489 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 22:53:51.018795 31489 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:53:51.018841 31489 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:53:51.018905 31489 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 22:53:51.018956 31489 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 22:53:51.027959 31489 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.113.148:40873
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786315935617919-15814-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786315935617919-15814-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.113.190:33143
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.113.190:33143
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786315935617919-15814-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.113.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786315935617919-15814-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:53:51.029194 31489 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:53:51.030952 31489 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:53:51.042161 31495 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:53:51.042269 31497 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:53:51.042229 31500 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:53:51.043020 31489 server_base.cc:1034] running on GCE node
I20260809 22:53:51.043704 31489 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:53:51.045022 31489 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:53:51.046180 31489 hybrid_clock.cc:648] HybridClock initialized: now 1786316031046135 us; error 39 us; skew 500 ppm
I20260809 22:53:51.046501 31489 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 22:53:51.048570 31489 webserver.cc:460] Webserver started at http://127.15.113.190:34549/ using document root <none> and password file <none>
I20260809 22:53:51.049266 31489 fs_manager.cc:359] Metadata directory not provided
I20260809 22:53:51.049361 31489 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:53:51.049669 31489 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:53:51.052417 31489 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786315935617919-15814-0/minicluster-data/master-0/data/instance:
uuid: "e84ff0bcf7db4222a4534bcef64caa5f"
format_stamp: "Formatted at 2026-08-09 22:53:51 on dist-test-slave-ncp9"
server_key: "8e3eba35051ae2600ded63b08df2d418"
server_key_iv: "ef56bda90c46b2db431409c22f430525"
server_key_version: "encryptionkey@0"
I20260809 22:53:51.053217 31489 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786315935617919-15814-0/minicluster-data/master-0/wal/instance:
uuid: "e84ff0bcf7db4222a4534bcef64caa5f"
format_stamp: "Formatted at 2026-08-09 22:53:51 on dist-test-slave-ncp9"
server_key: "8e3eba35051ae2600ded63b08df2d418"
server_key_iv: "ef56bda90c46b2db431409c22f430525"
server_key_version: "encryptionkey@0"
I20260809 22:53:51.058032 31489 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260809 22:53:51.061203 31512 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:53:51.062439 31489 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260809 22:53:51.062588 31489 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786315935617919-15814-0/minicluster-data/master-0/data,/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786315935617919-15814-0/minicluster-data/master-0/wal
uuid: "e84ff0bcf7db4222a4534bcef64caa5f"
format_stamp: "Formatted at 2026-08-09 22:53:51 on dist-test-slave-ncp9"
server_key: "8e3eba35051ae2600ded63b08df2d418"
server_key_iv: "ef56bda90c46b2db431409c22f430525"
server_key_version: "encryptionkey@0"
I20260809 22:53:51.062776 31489 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786315935617919-15814-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786315935617919-15814-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786315935617919-15814-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:53:51.068121 31489 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:53:51.068830 31489 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:53:51.069145 31489 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:53:51.085310 31489 rpc_server.cc:307] RPC server started. Bound to: 127.15.113.190:33143
I20260809 22:53:51.085352 31588 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.113.190:33143 every 8 connection(s)
I20260809 22:53:51.086517 31489 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786315935617919-15814-0/minicluster-data/master-0/data/info.pb
I20260809 22:53:51.087016 15814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskItYPJR/build/asan/bin/kudu as pid 31489
I20260809 22:53:51.087220 15814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786315935617919-15814-0/minicluster-data/master-0/wal/instance
I20260809 22:53:51.087982 15814 external_mini_cluster.cc:1428] Setting key a414901f2f30c84a27c7499aa7d8fe32
I20260809 22:53:51.090581 31589 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:53:51.095878 31593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:51.089896 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44322 (local address 127.15.113.190:33143)
0809 22:53:51.090923 (+ 1027us) server_negotiation.cc:206] Beginning negotiation
0809 22:53:51.090933 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 22:53:51.090976 (+ 43us) server_negotiation.cc:407] Connection header received
0809 22:53:51.092515 (+ 1539us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:53:51.092536 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:53:51.092967 (+ 431us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:53:51.093291 (+ 324us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:53:51.094350 (+ 1059us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:53:51.094356 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:53:51.094369 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 22:53:51.094382 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:53:51.094471 (+ 89us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 22:53:51.094487 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:53:51.094668 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0809 22:53:51.094759 (+ 91us) server_negotiation.cc:299] Negotiation successful
0809 22:53:51.095007 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":585,"thread_start_us":356,"threads_started":1}
I20260809 22:53:51.101822 31589 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e84ff0bcf7db4222a4534bcef64caa5f: Bootstrap starting.
I20260809 22:53:51.104774 31589 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e84ff0bcf7db4222a4534bcef64caa5f: Neither blocks nor log segments found. Creating new log.
I20260809 22:53:51.105798 31589 log.cc:824] T 00000000000000000000000000000000 P e84ff0bcf7db4222a4534bcef64caa5f: Log is configured to *not* fsync() on all Append() calls
I20260809 22:53:51.108119 31589 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e84ff0bcf7db4222a4534bcef64caa5f: No bootstrap required, opened a new log
I20260809 22:53:51.115198 31589 raft_consensus.cc:348] T 00000000000000000000000000000000 P e84ff0bcf7db4222a4534bcef64caa5f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e84ff0bcf7db4222a4534bcef64caa5f" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 33143 } }
I20260809 22:53:51.115644 31589 raft_consensus.cc:374] T 00000000000000000000000000000000 P e84ff0bcf7db4222a4534bcef64caa5f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:53:51.115751 31589 raft_consensus.cc:729] T 00000000000000000000000000000000 P e84ff0bcf7db4222a4534bcef64caa5f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e84ff0bcf7db4222a4534bcef64caa5f, State: Initialized, Role: FOLLOWER
I20260809 22:53:51.116345 31589 consensus_queue.cc:260] T 00000000000000000000000000000000 P e84ff0bcf7db4222a4534bcef64caa5f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e84ff0bcf7db4222a4534bcef64caa5f" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 33143 } }
I20260809 22:53:51.116567 31589 raft_consensus.cc:388] T 00000000000000000000000000000000 P e84ff0bcf7db4222a4534bcef64caa5f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 22:53:51.116668 31589 raft_consensus.cc:482] T 00000000000000000000000000000000 P e84ff0bcf7db4222a4534bcef64caa5f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 22:53:51.116784 31589 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e84ff0bcf7db4222a4534bcef64caa5f [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:53:51.118791 31589 raft_consensus.cc:504] T 00000000000000000000000000000000 P e84ff0bcf7db4222a4534bcef64caa5f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e84ff0bcf7db4222a4534bcef64caa5f" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 33143 } }
I20260809 22:53:51.119220 31589 leader_election.cc:302] T 00000000000000000000000000000000 P e84ff0bcf7db4222a4534bcef64caa5f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e84ff0bcf7db4222a4534bcef64caa5f; no voters:
I20260809 22:53:51.119742 31589 leader_election.cc:288] T 00000000000000000000000000000000 P e84ff0bcf7db4222a4534bcef64caa5f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 22:53:51.119848 31597 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e84ff0bcf7db4222a4534bcef64caa5f [term 1 FOLLOWER]: Leader election won for term 1
I20260809 22:53:51.120489 31597 raft_consensus.cc:686] T 00000000000000000000000000000000 P e84ff0bcf7db4222a4534bcef64caa5f [term 1 LEADER]: Becoming Leader. State: Replica: e84ff0bcf7db4222a4534bcef64caa5f, State: Running, Role: LEADER
I20260809 22:53:51.121047 31597 consensus_queue.cc:237] T 00000000000000000000000000000000 P e84ff0bcf7db4222a4534bcef64caa5f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e84ff0bcf7db4222a4534bcef64caa5f" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 33143 } }
I20260809 22:53:51.121671 31589 sys_catalog.cc:564] T 00000000000000000000000000000000 P e84ff0bcf7db4222a4534bcef64caa5f [sys.catalog]: configured and running, proceeding with master startup.
I20260809 22:53:51.123864 31598 sys_catalog.cc:455] T 00000000000000000000000000000000 P e84ff0bcf7db4222a4534bcef64caa5f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e84ff0bcf7db4222a4534bcef64caa5f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e84ff0bcf7db4222a4534bcef64caa5f" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 33143 } } }
I20260809 22:53:51.124293 31598 sys_catalog.cc:458] T 00000000000000000000000000000000 P e84ff0bcf7db4222a4534bcef64caa5f [sys.catalog]: This master's current role is: LEADER
I20260809 22:53:51.124766 31603 sys_catalog.cc:455] T 00000000000000000000000000000000 P e84ff0bcf7db4222a4534bcef64caa5f [sys.catalog]: SysCatalogTable state changed. Reason: New leader e84ff0bcf7db4222a4534bcef64caa5f. Latest consensus state: current_term: 1 leader_uuid: "e84ff0bcf7db4222a4534bcef64caa5f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e84ff0bcf7db4222a4534bcef64caa5f" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 33143 } } }
I20260809 22:53:51.125094 31603 sys_catalog.cc:458] T 00000000000000000000000000000000 P e84ff0bcf7db4222a4534bcef64caa5f [sys.catalog]: This master's current role is: LEADER
I20260809 22:53:51.126308 31611 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 22:53:51.130280 31611 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 22:53:51.138973 31611 catalog_manager.cc:1347] Generated new cluster ID: 7ed34957b424466fa96e1e0cddae2cbf
I20260809 22:53:51.139081 31611 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 22:53:51.159293 31611 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 22:53:51.159675 31611 catalog_manager.cc:1504] Loading token signing keys...
I20260809 22:53:51.166155 31611 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e84ff0bcf7db4222a4534bcef64caa5f: Generated new TSK 0
I20260809 22:53:51.166986 31611 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 22:53:51.181156 15814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskItYPJR/build/asan/bin/kudu
/tmp/dist-test-taskItYPJR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786315935617919-15814-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786315935617919-15814-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786315935617919-15814-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786315935617919-15814-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.113.129:0
--local_ip_for_outbound_sockets=127.15.113.129
--webserver_interface=127.15.113.129
--webserver_port=0
--tserver_master_addrs=127.15.113.190:33143
--builtin_ntp_servers=127.15.113.148:40873
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260809 22:53:51.519229 31625 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:53:51.519502 31625 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:53:51.519640 31625 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:53:51.527006 31625 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:53:51.527100 31625 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:53:51.527233 31625 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.113.129
I20260809 22:53:51.536643 31625 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.113.148:40873
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786315935617919-15814-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786315935617919-15814-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.113.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786315935617919-15814-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.113.129
--webserver_port=0
--tserver_master_addrs=127.15.113.190:33143
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.113.129
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786315935617919-15814-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:53:51.537976 31625 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:53:51.539738 31625 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:53:51.552299 31632 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:53:51.552338 31636 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:53:51.552439 31633 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:53:51.553061 31625 server_base.cc:1034] running on GCE node
I20260809 22:53:51.553634 31625 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:53:51.554486 31625 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:53:51.555681 31625 hybrid_clock.cc:648] HybridClock initialized: now 1786316031555633 us; error 37 us; skew 500 ppm
I20260809 22:53:51.555974 31625 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 22:53:51.558082 31625 webserver.cc:460] Webserver started at http://127.15.113.129:37465/ using document root <none> and password file <none>
I20260809 22:53:51.558723 31625 fs_manager.cc:359] Metadata directory not provided
I20260809 22:53:51.558825 31625 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:53:51.559121 31625 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:53:51.561992 31625 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786315935617919-15814-0/minicluster-data/ts-0/data/instance:
uuid: "efd1c1321bcc4c01af0e0453422101a1"
format_stamp: "Formatted at 2026-08-09 22:53:51 on dist-test-slave-ncp9"
server_key: "68b0e27a07fa0d7e580107ac77191e39"
server_key_iv: "61b2bd8be724e33da818965c125080b8"
server_key_version: "encryptionkey@0"
I20260809 22:53:51.562837 31625 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786315935617919-15814-0/minicluster-data/ts-0/wal/instance:
uuid: "efd1c1321bcc4c01af0e0453422101a1"
format_stamp: "Formatted at 2026-08-09 22:53:51 on dist-test-slave-ncp9"
server_key: "68b0e27a07fa0d7e580107ac77191e39"
server_key_iv: "61b2bd8be724e33da818965c125080b8"
server_key_version: "encryptionkey@0"
I20260809 22:53:51.567991 31625 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260809 22:53:51.571288 31649 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:53:51.572610 31625 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 22:53:51.572764 31625 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786315935617919-15814-0/minicluster-data/ts-0/data,/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786315935617919-15814-0/minicluster-data/ts-0/wal
uuid: "efd1c1321bcc4c01af0e0453422101a1"
format_stamp: "Formatted at 2026-08-09 22:53:51 on dist-test-slave-ncp9"
server_key: "68b0e27a07fa0d7e580107ac77191e39"
server_key_iv: "61b2bd8be724e33da818965c125080b8"
server_key_version: "encryptionkey@0"
I20260809 22:53:51.572980 31625 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786315935617919-15814-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786315935617919-15814-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786315935617919-15814-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:53:51.578251 31625 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:53:51.578985 31625 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:53:51.579221 31625 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:53:51.580077 31625 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:53:51.581405 31625 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:53:51.581501 31625 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:53:51.581604 31625 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:53:51.581682 31625 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:53:51.611009 31625 rpc_server.cc:307] RPC server started. Bound to: 127.15.113.129:36985
I20260809 22:53:51.611058 31829 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.113.129:36985 every 8 connection(s)
I20260809 22:53:51.612291 31625 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786315935617919-15814-0/minicluster-data/ts-0/data/info.pb
I20260809 22:53:51.617949 15814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskItYPJR/build/asan/bin/kudu as pid 31625
I20260809 22:53:51.618157 15814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786315935617919-15814-0/minicluster-data/ts-0/wal/instance
I20260809 22:53:51.618777 15814 external_mini_cluster.cc:1428] Setting key 429ac8502dd02754722b2d865d333413
I20260809 22:53:51.620343 31832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:51.614788 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.113.129:33319 (local address 127.15.113.190:33143)
0809 22:53:51.615213 (+ 425us) server_negotiation.cc:206] Beginning negotiation
0809 22:53:51.615217 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 22:53:51.616341 (+ 1124us) server_negotiation.cc:407] Connection header received
0809 22:53:51.618085 (+ 1744us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:53:51.618091 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:53:51.618153 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:53:51.618251 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:53:51.619600 (+ 1349us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:53:51.619605 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:53:51.619610 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 22:53:51.619631 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:53:51.619683 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 22:53:51.619699 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:53:51.619821 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0809 22:53:51.620064 (+ 243us) server_negotiation.cc:299] Negotiation successful
0809 22:53:51.620114 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":323,"thread_start_us":224,"threads_started":1}
I20260809 22:53:51.621438 31833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:51.615273 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:33143 (local address 127.15.113.129:33319)
0809 22:53:51.616181 (+ 908us) negotiation.cc:107] Waiting for socket to connect
0809 22:53:51.616217 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0809 22:53:51.617806 (+ 1589us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:53:51.618503 (+ 697us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:53:51.618511 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:53:51.618601 (+ 90us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:53:51.619229 (+ 628us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 22:53:51.619232 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:53:51.619284 (+ 52us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 22:53:51.619399 (+ 115us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:53:51.619842 (+ 443us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:53:51.619855 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:53:51.619864 (+ 9us) client_negotiation.cc:769] Sending connection context
0809 22:53:51.620155 (+ 291us) client_negotiation.cc:240] Negotiation successful
0809 22:53:51.620409 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":442,"thread_start_us":264,"threads_started":1}
I20260809 22:53:51.621822 15814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskItYPJR/build/asan/bin/kudu
/tmp/dist-test-taskItYPJR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786315935617919-15814-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786315935617919-15814-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786315935617919-15814-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786315935617919-15814-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.113.130:0
--local_ip_for_outbound_sockets=127.15.113.130
--webserver_interface=127.15.113.130
--webserver_port=0
--tserver_master_addrs=127.15.113.190:33143
--builtin_ntp_servers=127.15.113.148:40873
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260809 22:53:51.622926 31830 heartbeater.cc:347] Connected to a master server at 127.15.113.190:33143
I20260809 22:53:51.623263 31830 heartbeater.cc:464] Registering TS with master...
I20260809 22:53:51.623948 31830 heartbeater.cc:511] Master 127.15.113.190:33143 requested a full tablet report, sending...
I20260809 22:53:51.626368 31536 ts_manager.cc:194] Registered new tserver with Master: efd1c1321bcc4c01af0e0453422101a1 (127.15.113.129:36985)
W20260809 22:53:51.953099 31834 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:53:51.953380 31834 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:53:51.953537 31834 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:53:51.960731 31834 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:53:51.960831 31834 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:53:51.961016 31834 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.113.130
I20260809 22:53:51.970124 31834 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.113.148:40873
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786315935617919-15814-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786315935617919-15814-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.113.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786315935617919-15814-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.113.130
--webserver_port=0
--tserver_master_addrs=127.15.113.190:33143
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.113.130
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786315935617919-15814-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:53:51.971465 31834 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:53:51.973327 31834 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:53:51.986447 31842 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:53:51.986413 31844 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:53:51.986413 31847 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:53:51.987201 31834 server_base.cc:1034] running on GCE node
I20260809 22:53:51.987769 31834 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:53:51.988660 31834 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:53:51.989858 31834 hybrid_clock.cc:648] HybridClock initialized: now 1786316031989805 us; error 49 us; skew 500 ppm
I20260809 22:53:51.990168 31834 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 22:53:51.992300 31834 webserver.cc:460] Webserver started at http://127.15.113.130:33823/ using document root <none> and password file <none>
I20260809 22:53:51.993005 31834 fs_manager.cc:359] Metadata directory not provided
I20260809 22:53:51.993108 31834 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:53:51.993433 31834 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:53:51.996173 31834 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786315935617919-15814-0/minicluster-data/ts-1/data/instance:
uuid: "b86939425f8f4cf0bbd72374f7d80b03"
format_stamp: "Formatted at 2026-08-09 22:53:51 on dist-test-slave-ncp9"
server_key: "c53fc4b05117170d822dc35ed8050c28"
server_key_iv: "171d2555941f40bdcee510bbad31c04a"
server_key_version: "encryptionkey@0"
I20260809 22:53:51.996990 31834 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786315935617919-15814-0/minicluster-data/ts-1/wal/instance:
uuid: "b86939425f8f4cf0bbd72374f7d80b03"
format_stamp: "Formatted at 2026-08-09 22:53:51 on dist-test-slave-ncp9"
server_key: "c53fc4b05117170d822dc35ed8050c28"
server_key_iv: "171d2555941f40bdcee510bbad31c04a"
server_key_version: "encryptionkey@0"
I20260809 22:53:52.001885 31834 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260809 22:53:52.005138 31854 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:53:52.006408 31834 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 22:53:52.006556 31834 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786315935617919-15814-0/minicluster-data/ts-1/data,/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786315935617919-15814-0/minicluster-data/ts-1/wal
uuid: "b86939425f8f4cf0bbd72374f7d80b03"
format_stamp: "Formatted at 2026-08-09 22:53:51 on dist-test-slave-ncp9"
server_key: "c53fc4b05117170d822dc35ed8050c28"
server_key_iv: "171d2555941f40bdcee510bbad31c04a"
server_key_version: "encryptionkey@0"
I20260809 22:53:52.006757 31834 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786315935617919-15814-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786315935617919-15814-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786315935617919-15814-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:53:52.012087 31834 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:53:52.012805 31834 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:53:52.013077 31834 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:53:52.013932 31834 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:53:52.015307 31834 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:53:52.015399 31834 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:53:52.015507 31834 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:53:52.015578 31834 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:53:52.045571 31834 rpc_server.cc:307] RPC server started. Bound to: 127.15.113.130:43517
I20260809 22:53:52.045637 32017 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.113.130:43517 every 8 connection(s)
I20260809 22:53:52.046802 31834 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786315935617919-15814-0/minicluster-data/ts-1/data/info.pb
I20260809 22:53:52.050904 15814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskItYPJR/build/asan/bin/kudu as pid 31834
I20260809 22:53:52.051105 15814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786315935617919-15814-0/minicluster-data/ts-1/wal/instance
I20260809 22:53:52.051671 15814 external_mini_cluster.cc:1428] Setting key ef15ee9a7b3d3d27a807e974f22f2602
I20260809 22:53:52.054702 15814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskItYPJR/build/asan/bin/kudu
/tmp/dist-test-taskItYPJR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786315935617919-15814-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786315935617919-15814-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786315935617919-15814-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786315935617919-15814-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.113.131:0
--local_ip_for_outbound_sockets=127.15.113.131
--webserver_interface=127.15.113.131
--webserver_port=0
--tserver_master_addrs=127.15.113.190:33143
--builtin_ntp_servers=127.15.113.148:40873
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260809 22:53:52.055104 31832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:52.049242 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.113.130:34135 (local address 127.15.113.190:33143)
0809 22:53:52.049407 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0809 22:53:52.049412 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:53:52.050903 (+ 1491us) server_negotiation.cc:407] Connection header received
0809 22:53:52.052973 (+ 2070us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:53:52.052978 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:53:52.053054 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:53:52.053152 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:53:52.054403 (+ 1251us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:53:52.054408 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:53:52.054411 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 22:53:52.054426 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:53:52.054460 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 22:53:52.054471 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:53:52.054556 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0809 22:53:52.054791 (+ 235us) server_negotiation.cc:299] Negotiation successful
0809 22:53:52.054857 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260809 22:53:52.056219 32020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:52.049733 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:33143 (local address 127.15.113.130:34135)
0809 22:53:52.050716 (+ 983us) negotiation.cc:107] Waiting for socket to connect
0809 22:53:52.050776 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0809 22:53:52.052700 (+ 1924us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:53:52.053337 (+ 637us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:53:52.053346 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:53:52.053431 (+ 85us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:53:52.054066 (+ 635us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 22:53:52.054069 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:53:52.054118 (+ 49us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 22:53:52.054231 (+ 113us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:53:52.054570 (+ 339us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:53:52.054582 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:53:52.054589 (+ 7us) client_negotiation.cc:769] Sending connection context
0809 22:53:52.054892 (+ 303us) client_negotiation.cc:240] Negotiation successful
0809 22:53:52.055129 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":449,"thread_start_us":259,"threads_started":1}
I20260809 22:53:52.057482 32018 heartbeater.cc:347] Connected to a master server at 127.15.113.190:33143
I20260809 22:53:52.057789 32018 heartbeater.cc:464] Registering TS with master...
I20260809 22:53:52.058413 32018 heartbeater.cc:511] Master 127.15.113.190:33143 requested a full tablet report, sending...
I20260809 22:53:52.059758 31536 ts_manager.cc:194] Registered new tserver with Master: b86939425f8f4cf0bbd72374f7d80b03 (127.15.113.130:43517)
W20260809 22:53:52.386682 32021 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:53:52.386929 32021 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:53:52.387022 32021 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:53:52.393857 32021 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:53:52.393940 32021 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:53:52.394052 32021 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.113.131
I20260809 22:53:52.402665 32021 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.113.148:40873
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786315935617919-15814-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786315935617919-15814-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.113.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786315935617919-15814-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.113.131
--webserver_port=0
--tserver_master_addrs=127.15.113.190:33143
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.113.131
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786315935617919-15814-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:53:52.403775 32021 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:53:52.405411 32021 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:53:52.417534 32029 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:53:52.417599 32031 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:53:52.417500 32033 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:53:52.418617 32021 server_base.cc:1034] running on GCE node
I20260809 22:53:52.419160 32021 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:53:52.420002 32021 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:53:52.421196 32021 hybrid_clock.cc:648] HybridClock initialized: now 1786316032421157 us; error 35 us; skew 500 ppm
I20260809 22:53:52.421478 32021 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 22:53:52.423537 32021 webserver.cc:460] Webserver started at http://127.15.113.131:46495/ using document root <none> and password file <none>
I20260809 22:53:52.424189 32021 fs_manager.cc:359] Metadata directory not provided
I20260809 22:53:52.424279 32021 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:53:52.424583 32021 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:53:52.427294 32021 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786315935617919-15814-0/minicluster-data/ts-2/data/instance:
uuid: "af2ffe7d23644b69a8bc058df0ae702d"
format_stamp: "Formatted at 2026-08-09 22:53:52 on dist-test-slave-ncp9"
server_key: "a3a53ba193cf3bc86a0a4a6d1e8f8b92"
server_key_iv: "0e1e306eba2c2090c6719da2972393c5"
server_key_version: "encryptionkey@0"
I20260809 22:53:52.428020 32021 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786315935617919-15814-0/minicluster-data/ts-2/wal/instance:
uuid: "af2ffe7d23644b69a8bc058df0ae702d"
format_stamp: "Formatted at 2026-08-09 22:53:52 on dist-test-slave-ncp9"
server_key: "a3a53ba193cf3bc86a0a4a6d1e8f8b92"
server_key_iv: "0e1e306eba2c2090c6719da2972393c5"
server_key_version: "encryptionkey@0"
I20260809 22:53:52.432770 32021 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.002s
I20260809 22:53:52.435943 32043 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:53:52.437219 32021 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 22:53:52.437366 32021 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786315935617919-15814-0/minicluster-data/ts-2/data,/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786315935617919-15814-0/minicluster-data/ts-2/wal
uuid: "af2ffe7d23644b69a8bc058df0ae702d"
format_stamp: "Formatted at 2026-08-09 22:53:52 on dist-test-slave-ncp9"
server_key: "a3a53ba193cf3bc86a0a4a6d1e8f8b92"
server_key_iv: "0e1e306eba2c2090c6719da2972393c5"
server_key_version: "encryptionkey@0"
I20260809 22:53:52.437562 32021 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786315935617919-15814-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786315935617919-15814-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786315935617919-15814-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:53:52.442857 32021 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:53:52.443568 32021 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:53:52.443800 32021 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:53:52.444669 32021 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:53:52.446012 32021 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:53:52.446105 32021 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:53:52.446218 32021 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:53:52.446290 32021 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:53:52.476413 32021 rpc_server.cc:307] RPC server started. Bound to: 127.15.113.131:36397
I20260809 22:53:52.476451 32200 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.113.131:36397 every 8 connection(s)
I20260809 22:53:52.477788 32021 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786315935617919-15814-0/minicluster-data/ts-2/data/info.pb
I20260809 22:53:52.481823 15814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskItYPJR/build/asan/bin/kudu as pid 32021
I20260809 22:53:52.482004 15814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786315935617919-15814-0/minicluster-data/ts-2/wal/instance
I20260809 22:53:52.482445 15814 external_mini_cluster.cc:1428] Setting key 898f118bb9e511e24020604734a5a1b8
I20260809 22:53:52.485805 31832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:52.480255 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.113.131:50873 (local address 127.15.113.190:33143)
0809 22:53:52.480438 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0809 22:53:52.480443 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:53:52.481855 (+ 1412us) server_negotiation.cc:407] Connection header received
0809 22:53:52.483613 (+ 1758us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:53:52.483619 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:53:52.483687 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:53:52.483776 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:53:52.485119 (+ 1343us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:53:52.485124 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:53:52.485129 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 22:53:52.485150 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:53:52.485190 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 22:53:52.485206 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:53:52.485302 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0809 22:53:52.485547 (+ 245us) server_negotiation.cc:299] Negotiation successful
0809 22:53:52.485605 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260809 22:53:52.486653 32205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:52.480763 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:33143 (local address 127.15.113.131:50873)
0809 22:53:52.481693 (+ 930us) negotiation.cc:107] Waiting for socket to connect
0809 22:53:52.481734 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0809 22:53:52.483321 (+ 1587us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:53:52.484011 (+ 690us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:53:52.484019 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:53:52.484106 (+ 87us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:53:52.484733 (+ 627us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 22:53:52.484737 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:53:52.484788 (+ 51us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 22:53:52.484927 (+ 139us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:53:52.485314 (+ 387us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:53:52.485331 (+ 17us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:53:52.485339 (+ 8us) client_negotiation.cc:769] Sending connection context
0809 22:53:52.485580 (+ 241us) client_negotiation.cc:240] Negotiation successful
0809 22:53:52.485767 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":470,"thread_start_us":293,"threads_started":1}
I20260809 22:53:52.487843 32201 heartbeater.cc:347] Connected to a master server at 127.15.113.190:33143
I20260809 22:53:52.488106 32201 heartbeater.cc:464] Registering TS with master...
I20260809 22:53:52.488660 32201 heartbeater.cc:511] Master 127.15.113.190:33143 requested a full tablet report, sending...
I20260809 22:53:52.489923 31536 ts_manager.cc:194] Registered new tserver with Master: af2ffe7d23644b69a8bc058df0ae702d (127.15.113.131:36397)
I20260809 22:53:52.497606 15814 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260809 22:53:52.503507 31832 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:53:52.501833 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44332 (local address 127.15.113.190:33143)
0809 22:53:52.501995 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0809 22:53:52.502000 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:53:52.502497 (+ 497us) server_negotiation.cc:407] Connection header received
0809 22:53:52.502767 (+ 270us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:53:52.502771 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:53:52.502821 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:53:52.502898 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:53:52.503267 (+ 369us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44332: BlockingRecv error: recv got EOF from 127.0.0.1:44332 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
I20260809 22:53:52.504258 15814 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskItYPJR/build/asan/bin/kudu pid 31625
I20260809 22:53:52.507558 32215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:52.505238 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48626 (local address 127.15.113.129:36985)
0809 22:53:52.505741 (+ 503us) server_negotiation.cc:206] Beginning negotiation
0809 22:53:52.505748 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:53:52.505772 (+ 24us) server_negotiation.cc:407] Connection header received
0809 22:53:52.505828 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:53:52.505834 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:53:52.506037 (+ 203us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:53:52.506157 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:53:52.506982 (+ 825us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:53:52.506988 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:53:52.506999 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 22:53:52.507016 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:53:52.507056 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 22:53:52.507069 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:53:52.507169 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0809 22:53:52.507253 (+ 84us) server_negotiation.cc:299] Negotiation successful
0809 22:53:52.507328 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":359,"thread_start_us":218,"threads_started":1}
I20260809 22:53:52.508698 31823 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:53:52.588614 15814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskItYPJR/build/asan/bin/kudu with pid 31625
I20260809 22:53:52.611315 15814 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskItYPJR/build/asan/bin/kudu pid 31834
I20260809 22:53:52.614790 32217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:52.612260 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38540 (local address 127.15.113.130:43517)
0809 22:53:52.612823 (+ 563us) server_negotiation.cc:206] Beginning negotiation
0809 22:53:52.612830 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:53:52.612858 (+ 28us) server_negotiation.cc:407] Connection header received
0809 22:53:52.612967 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:53:52.612976 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:53:52.613254 (+ 278us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:53:52.613388 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:53:52.614197 (+ 809us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:53:52.614204 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:53:52.614217 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 22:53:52.614236 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:53:52.614281 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 22:53:52.614296 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:53:52.614401 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0809 22:53:52.614456 (+ 55us) server_negotiation.cc:299] Negotiation successful
0809 22:53:52.614536 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":412,"thread_start_us":279,"threads_started":1}
I20260809 22:53:52.615994 32011 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:53:52.695449 15814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskItYPJR/build/asan/bin/kudu with pid 31834
I20260809 22:53:52.714210 15814 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskItYPJR/build/asan/bin/kudu pid 32021
I20260809 22:53:52.718019 32220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:52.715240 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50984 (local address 127.15.113.131:36397)
0809 22:53:52.715927 (+ 687us) server_negotiation.cc:206] Beginning negotiation
0809 22:53:52.715934 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:53:52.715963 (+ 29us) server_negotiation.cc:407] Connection header received
0809 22:53:52.716025 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:53:52.716031 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:53:52.716248 (+ 217us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:53:52.716373 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:53:52.717327 (+ 954us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:53:52.717334 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:53:52.717345 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 22:53:52.717362 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:53:52.717401 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 22:53:52.717415 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:53:52.717553 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0809 22:53:52.717604 (+ 51us) server_negotiation.cc:299] Negotiation successful
0809 22:53:52.717711 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":536,"thread_start_us":378,"threads_started":1}
I20260809 22:53:52.719264 32196 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:53:52.798861 15814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskItYPJR/build/asan/bin/kudu with pid 32021
I20260809 22:53:52.819957 15814 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskItYPJR/build/asan/bin/kudu pid 31489
I20260809 22:53:52.821175 31579 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:53:52.934126 15814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskItYPJR/build/asan/bin/kudu with pid 31489
2026-08-09T22:53:52Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (2335 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315935617919-15814-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 22:53:53 dist-test-slave-ncp9 krb5kdc[32227](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 22:53:53 dist-test-slave-ncp9 krb5kdc[32227](info): set up 2 sockets
Aug 09 22:53:53 dist-test-slave-ncp9 krb5kdc[32227](info): commencing operation
krb5kdc: starting...
W20260809 22:53:54.217451 15814 mini_kdc.cc:121] Time spent starting KDC: real 1.236s user 0.010s sys 0.028s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315935617919-15814-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315935617919-15814-0/krb5kdc/test-user.keytab.
Aug 09 22:53:54 dist-test-slave-ncp9 krb5kdc[32227](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316034, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T22:53:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T22:53:54Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.113.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.113.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.113.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.190.keytab.
Entry for principal kudu/127.15.113.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.190.keytab.
I20260809 22:53:54.648361 15814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskItYPJR/build/asan/bin/kudu
/tmp/dist-test-taskItYPJR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315935617919-15814-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315935617919-15814-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315935617919-15814-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315935617919-15814-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.113.190:44105
--webserver_interface=127.15.113.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.190.keytab
--principal=kudu/127.15.113.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.113.148:46473
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.113.190:44105
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315935617919-15814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315935617919-15814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315935617919-15814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:53:55.199735 32243 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:53:55.200270 32243 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:53:55.200474 32243 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:53:55.212440 32243 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 22:53:55.212595 32243 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:53:55.212682 32243 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:53:55.212785 32243 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 22:53:55.212857 32243 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 22:53:55.231856 32243 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.113.148:46473
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315935617919-15814-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315935617919-15814-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.113.190:44105
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.190.keytab
--principal=kudu/127.15.113.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.113.190:44105
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315935617919-15814-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.113.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315935617919-15814-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:53:55.234053 32243 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:53:55.236192 32243 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:53:55.255477 32251 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:53:55.255515 32252 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:53:55.255908 32254 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:53:55.257392 32243 server_base.cc:1034] running on GCE node
I20260809 22:53:55.258853 32243 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:53:55.261644 32243 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:53:55.262981 32243 hybrid_clock.cc:648] HybridClock initialized: now 1786316035262864 us; error 80 us; skew 500 ppm
I20260809 22:53:55.263579 32243 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:53:55 dist-test-slave-ncp9 krb5kdc[32227](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316035, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.113.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:53:55.273470 32243 init.cc:377] Logged in from keytab as kudu/127.15.113.190@KRBTEST.COM (short username kudu)
I20260809 22:53:55.276027 32243 webserver.cc:460] Webserver started at http://127.15.113.190:33827/ using document root <none> and password file <none>
I20260809 22:53:55.277210 32243 fs_manager.cc:359] Metadata directory not provided
I20260809 22:53:55.277416 32243 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:53:55.277946 32243 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:53:55.283761 32243 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315935617919-15814-0/minicluster-data/master-0/data/instance:
uuid: "ac5bc82598e64cdd856390462e434519"
format_stamp: "Formatted at 2026-08-09 22:53:55 on dist-test-slave-ncp9"
server_key: "385ad31fc4f332299d611f71141dda1a"
server_key_iv: "e3b1ea19b39cda74368612c417f1bf75"
server_key_version: "encryptionkey@0"
I20260809 22:53:55.285329 32243 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315935617919-15814-0/minicluster-data/master-0/wal/instance:
uuid: "ac5bc82598e64cdd856390462e434519"
format_stamp: "Formatted at 2026-08-09 22:53:55 on dist-test-slave-ncp9"
server_key: "385ad31fc4f332299d611f71141dda1a"
server_key_iv: "e3b1ea19b39cda74368612c417f1bf75"
server_key_version: "encryptionkey@0"
I20260809 22:53:55.295259 32243 fs_manager.cc:691] Time spent creating directory manager: real 0.009s user 0.008s sys 0.001s
I20260809 22:53:55.302315 32261 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:53:55.304839 32243 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.005s sys 0.000s
I20260809 22:53:55.305150 32243 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315935617919-15814-0/minicluster-data/master-0/data,/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315935617919-15814-0/minicluster-data/master-0/wal
uuid: "ac5bc82598e64cdd856390462e434519"
format_stamp: "Formatted at 2026-08-09 22:53:55 on dist-test-slave-ncp9"
server_key: "385ad31fc4f332299d611f71141dda1a"
server_key_iv: "e3b1ea19b39cda74368612c417f1bf75"
server_key_version: "encryptionkey@0"
I20260809 22:53:55.305492 32243 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315935617919-15814-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315935617919-15814-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315935617919-15814-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:53:55.330672 32243 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:53:55.338055 32243 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:53:55.338596 32243 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:53:55.370313 32243 rpc_server.cc:307] RPC server started. Bound to: 127.15.113.190:44105
I20260809 22:53:55.370340 32347 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.113.190:44105 every 8 connection(s)
I20260809 22:53:55.372915 32243 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315935617919-15814-0/minicluster-data/master-0/data/info.pb
I20260809 22:53:55.376659 15814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskItYPJR/build/asan/bin/kudu as pid 32243
I20260809 22:53:55.377108 15814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315935617919-15814-0/minicluster-data/master-0/wal/instance
I20260809 22:53:55.378346 15814 external_mini_cluster.cc:1428] Setting key 1270f935eed91803b74b355b3e37f030
I20260809 22:53:55.380401 32348 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:53:55.404155 32348 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ac5bc82598e64cdd856390462e434519: Bootstrap starting.
Aug 09 22:53:55 dist-test-slave-ncp9 krb5kdc[32227](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316034, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.113.190@KRBTEST.COM
I20260809 22:53:55.409662 32348 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ac5bc82598e64cdd856390462e434519: Neither blocks nor log segments found. Creating new log.
I20260809 22:53:55.411278 32348 log.cc:824] T 00000000000000000000000000000000 P ac5bc82598e64cdd856390462e434519: Log is configured to *not* fsync() on all Append() calls
I20260809 22:53:55.416445 32348 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ac5bc82598e64cdd856390462e434519: No bootstrap required, opened a new log
I20260809 22:53:55.417842 32352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:55.381934 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47994 (local address 127.15.113.190:44105)
0809 22:53:55.383270 (+ 1336us) server_negotiation.cc:206] Beginning negotiation
0809 22:53:55.383289 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0809 22:53:55.383367 (+ 78us) server_negotiation.cc:407] Connection header received
0809 22:53:55.387586 (+ 4219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:53:55.387640 (+ 54us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:53:55.388303 (+ 663us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:53:55.388806 (+ 503us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:53:55.392035 (+ 3229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:55.393959 (+ 1924us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:55.397117 (+ 3158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:55.398266 (+ 1149us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:55.398470 (+ 204us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:55.406473 (+ 8003us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:53:55.406492 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:53:55.406519 (+ 27us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:53:55.406592 (+ 73us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:53:55.411272 (+ 4680us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:53:55.412380 (+ 1108us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:53:55.412394 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:53:55.412402 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:53:55.412673 (+ 271us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:53:55.413428 (+ 755us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:53:55.413434 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:53:55.413445 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:53:55.414359 (+ 914us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:53:55.414698 (+ 339us) server_negotiation.cc:1035] Waiting for connection context
0809 22:53:55.415468 (+ 770us) server_negotiation.cc:299] Negotiation successful
0809 22:53:55.415905 (+ 437us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":956,"thread_start_us":554,"threads_started":1}
I20260809 22:53:55.428231 32348 raft_consensus.cc:348] T 00000000000000000000000000000000 P ac5bc82598e64cdd856390462e434519 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac5bc82598e64cdd856390462e434519" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 44105 } }
I20260809 22:53:55.428771 32348 raft_consensus.cc:374] T 00000000000000000000000000000000 P ac5bc82598e64cdd856390462e434519 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:53:55.428925 32348 raft_consensus.cc:729] T 00000000000000000000000000000000 P ac5bc82598e64cdd856390462e434519 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ac5bc82598e64cdd856390462e434519, State: Initialized, Role: FOLLOWER
I20260809 22:53:55.429858 32348 consensus_queue.cc:260] T 00000000000000000000000000000000 P ac5bc82598e64cdd856390462e434519 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac5bc82598e64cdd856390462e434519" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 44105 } }
I20260809 22:53:55.430176 32348 raft_consensus.cc:388] T 00000000000000000000000000000000 P ac5bc82598e64cdd856390462e434519 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 22:53:55.430320 32348 raft_consensus.cc:482] T 00000000000000000000000000000000 P ac5bc82598e64cdd856390462e434519 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 22:53:55.430493 32348 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ac5bc82598e64cdd856390462e434519 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:53:55.433586 32348 raft_consensus.cc:504] T 00000000000000000000000000000000 P ac5bc82598e64cdd856390462e434519 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac5bc82598e64cdd856390462e434519" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 44105 } }
I20260809 22:53:55.434227 32348 leader_election.cc:302] T 00000000000000000000000000000000 P ac5bc82598e64cdd856390462e434519 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ac5bc82598e64cdd856390462e434519; no voters:
I20260809 22:53:55.435231 32348 leader_election.cc:288] T 00000000000000000000000000000000 P ac5bc82598e64cdd856390462e434519 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 22:53:55.435421 32355 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ac5bc82598e64cdd856390462e434519 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 22:53:55.436292 32355 raft_consensus.cc:686] T 00000000000000000000000000000000 P ac5bc82598e64cdd856390462e434519 [term 1 LEADER]: Becoming Leader. State: Replica: ac5bc82598e64cdd856390462e434519, State: Running, Role: LEADER
I20260809 22:53:55.437139 32355 consensus_queue.cc:237] T 00000000000000000000000000000000 P ac5bc82598e64cdd856390462e434519 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac5bc82598e64cdd856390462e434519" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 44105 } }
I20260809 22:53:55.438216 32348 sys_catalog.cc:564] T 00000000000000000000000000000000 P ac5bc82598e64cdd856390462e434519 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 22:53:55.442031 32356 sys_catalog.cc:455] T 00000000000000000000000000000000 P ac5bc82598e64cdd856390462e434519 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ac5bc82598e64cdd856390462e434519" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac5bc82598e64cdd856390462e434519" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 44105 } } }
I20260809 22:53:55.442786 32356 sys_catalog.cc:458] T 00000000000000000000000000000000 P ac5bc82598e64cdd856390462e434519 [sys.catalog]: This master's current role is: LEADER
I20260809 22:53:55.442243 32358 sys_catalog.cc:455] T 00000000000000000000000000000000 P ac5bc82598e64cdd856390462e434519 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ac5bc82598e64cdd856390462e434519. Latest consensus state: current_term: 1 leader_uuid: "ac5bc82598e64cdd856390462e434519" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac5bc82598e64cdd856390462e434519" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 44105 } } }
I20260809 22:53:55.449163 32358 sys_catalog.cc:458] T 00000000000000000000000000000000 P ac5bc82598e64cdd856390462e434519 [sys.catalog]: This master's current role is: LEADER
I20260809 22:53:55.461037 32371 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 22:53:55.470444 32371 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 22:53:55.484124 32371 catalog_manager.cc:1347] Generated new cluster ID: 4e408116526a43b0a55c3a8f6a38ba18
I20260809 22:53:55.484411 32371 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 22:53:55.504602 32371 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 22:53:55.506599 32371 catalog_manager.cc:1504] Loading token signing keys...
I20260809 22:53:55.514995 32371 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ac5bc82598e64cdd856390462e434519: Generated new TSK 0
I20260809 22:53:55.516721 32371 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.113.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.113.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.113.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.129.keytab.
Entry for principal kudu/127.15.113.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.129.keytab.
I20260809 22:53:55.609843 15814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskItYPJR/build/asan/bin/kudu
/tmp/dist-test-taskItYPJR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315935617919-15814-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315935617919-15814-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315935617919-15814-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315935617919-15814-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.113.129:0
--local_ip_for_outbound_sockets=127.15.113.129
--webserver_interface=127.15.113.129
--webserver_port=0
--tserver_master_addrs=127.15.113.190:44105
--keytab_file=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.129.keytab
--principal=kudu/127.15.113.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.113.148:46473
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315935617919-15814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315935617919-15814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315935617919-15814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:53:56.397670 32390 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:53:56.398116 32390 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:53:56.398344 32390 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:53:56.410426 32390 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:53:56.410645 32390 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:53:56.410967 32390 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.113.129
I20260809 22:53:56.427446 32390 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.113.148:46473
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315935617919-15814-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315935617919-15814-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.129.keytab
--principal=kudu/127.15.113.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.113.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315935617919-15814-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.113.129
--webserver_port=0
--tserver_master_addrs=127.15.113.190:44105
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.113.129
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315935617919-15814-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:53:56.429680 32390 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:53:56.431989 32390 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:53:56.454342 32400 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:53:56.454424 32399 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:53:56.454730 32402 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:53:56.455783 32390 server_base.cc:1034] running on GCE node
I20260809 22:53:56.456967 32390 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:53:56.458587 32390 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:53:56.459853 32390 hybrid_clock.cc:648] HybridClock initialized: now 1786316036459762 us; error 85 us; skew 500 ppm
I20260809 22:53:56.460440 32390 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:53:56 dist-test-slave-ncp9 krb5kdc[32227](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316036, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.113.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:53:56.465962 32390 init.cc:377] Logged in from keytab as kudu/127.15.113.129@KRBTEST.COM (short username kudu)
I20260809 22:53:56.468413 32390 webserver.cc:460] Webserver started at http://127.15.113.129:38835/ using document root <none> and password file <none>
I20260809 22:53:56.469640 32390 fs_manager.cc:359] Metadata directory not provided
I20260809 22:53:56.469872 32390 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:53:56.470474 32390 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:53:56.476096 32390 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315935617919-15814-0/minicluster-data/ts-0/data/instance:
uuid: "6d150f09be4a4c43b2cae6553da74969"
format_stamp: "Formatted at 2026-08-09 22:53:56 on dist-test-slave-ncp9"
server_key: "5abde9ff3bc04cb0fa1eddfa473676eb"
server_key_iv: "ef592f81823c42b5e9ef57b510515aff"
server_key_version: "encryptionkey@0"
I20260809 22:53:56.477687 32390 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315935617919-15814-0/minicluster-data/ts-0/wal/instance:
uuid: "6d150f09be4a4c43b2cae6553da74969"
format_stamp: "Formatted at 2026-08-09 22:53:56 on dist-test-slave-ncp9"
server_key: "5abde9ff3bc04cb0fa1eddfa473676eb"
server_key_iv: "ef592f81823c42b5e9ef57b510515aff"
server_key_version: "encryptionkey@0"
I20260809 22:53:56.487270 32390 fs_manager.cc:691] Time spent creating directory manager: real 0.009s user 0.001s sys 0.008s
I20260809 22:53:56.493364 32413 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:53:56.495430 32390 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260809 22:53:56.495656 32390 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315935617919-15814-0/minicluster-data/ts-0/data,/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315935617919-15814-0/minicluster-data/ts-0/wal
uuid: "6d150f09be4a4c43b2cae6553da74969"
format_stamp: "Formatted at 2026-08-09 22:53:56 on dist-test-slave-ncp9"
server_key: "5abde9ff3bc04cb0fa1eddfa473676eb"
server_key_iv: "ef592f81823c42b5e9ef57b510515aff"
server_key_version: "encryptionkey@0"
I20260809 22:53:56.495931 32390 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315935617919-15814-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315935617919-15814-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315935617919-15814-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:53:56.522727 32390 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:53:56.534060 32390 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:53:56.534631 32390 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:53:56.536320 32390 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:53:56.538944 32390 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:53:56.539168 32390 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:53:56.539553 32390 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:53:56.539707 32390 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:53:56.591099 32390 rpc_server.cc:307] RPC server started. Bound to: 127.15.113.129:40037
I20260809 22:53:56.591174 32572 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.113.129:40037 every 8 connection(s)
I20260809 22:53:56.593626 32390 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315935617919-15814-0/minicluster-data/ts-0/data/info.pb
I20260809 22:53:56.597070 15814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskItYPJR/build/asan/bin/kudu as pid 32390
I20260809 22:53:56.597522 15814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315935617919-15814-0/minicluster-data/ts-0/wal/instance
I20260809 22:53:56.599107 15814 external_mini_cluster.cc:1428] Setting key 7097c3d511ea669ad034f7d06d1c5cc1
Aug 09 22:53:56 dist-test-slave-ncp9 krb5kdc[32227](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316036, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.113.129@KRBTEST.COM for kudu/127.15.113.190@KRBTEST.COM
I20260809 22:53:56.638607 32575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:56.598686 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.113.129:57709 (local address 127.15.113.190:44105)
0809 22:53:56.599655 (+ 969us) server_negotiation.cc:206] Beginning negotiation
0809 22:53:56.599675 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0809 22:53:56.601393 (+ 1718us) server_negotiation.cc:407] Connection header received
0809 22:53:56.606091 (+ 4698us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:53:56.606104 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:53:56.606219 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:53:56.606339 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:53:56.609711 (+ 3372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:56.610862 (+ 1151us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:56.615222 (+ 4360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:56.615817 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:56.615953 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:56.623810 (+ 7857us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:53:56.623819 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:53:56.623824 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:53:56.623962 (+ 138us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:53:56.631398 (+ 7436us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:53:56.632843 (+ 1445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:53:56.632871 (+ 28us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:53:56.632875 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:53:56.633219 (+ 344us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:53:56.634170 (+ 951us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:53:56.634175 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:53:56.634177 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:53:56.634747 (+ 570us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:53:56.635076 (+ 329us) server_negotiation.cc:1035] Waiting for connection context
0809 22:53:56.636574 (+ 1498us) server_negotiation.cc:299] Negotiation successful
0809 22:53:56.637297 (+ 723us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":704,"thread_start_us":362,"threads_started":1}
I20260809 22:53:56.640981 32576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:56.599804 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:44105 (local address 127.15.113.129:57709)
0809 22:53:56.601138 (+ 1334us) negotiation.cc:107] Waiting for socket to connect
0809 22:53:56.601246 (+ 108us) client_negotiation.cc:174] Beginning negotiation
0809 22:53:56.605604 (+ 4358us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:53:56.606833 (+ 1229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:53:56.606870 (+ 37us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:53:56.608087 (+ 1217us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:53:56.609470 (+ 1383us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:53:56.609516 (+ 46us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:53:56.611098 (+ 1582us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:53:56.611130 (+ 32us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:53:56.615030 (+ 3900us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:53:56.615052 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:53:56.616116 (+ 1064us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:53:56.616136 (+ 20us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:53:56.616713 (+ 577us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:56.617968 (+ 1255us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:53:56.618065 (+ 97us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:53:56.623215 (+ 5150us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:53:56.631640 (+ 8425us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:53:56.631662 (+ 22us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:53:56.631677 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:53:56.632581 (+ 904us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:53:56.633433 (+ 852us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:53:56.633449 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:53:56.633463 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:53:56.633886 (+ 423us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:53:56.634991 (+ 1105us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:53:56.635015 (+ 24us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:53:56.635792 (+ 777us) client_negotiation.cc:769] Sending connection context
0809 22:53:56.636273 (+ 481us) client_negotiation.cc:240] Negotiation successful
0809 22:53:56.637927 (+ 1654us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":716,"thread_start_us":436,"threads_started":1}
I20260809 22:53:56.643266 32573 heartbeater.cc:347] Connected to a master server at 127.15.113.190:44105
I20260809 22:53:56.643848 32573 heartbeater.cc:464] Registering TS with master...
I20260809 22:53:56.645437 32573 heartbeater.cc:511] Master 127.15.113.190:44105 requested a full tablet report, sending...
I20260809 22:53:56.649801 32290 ts_manager.cc:194] Registered new tserver with Master: 6d150f09be4a4c43b2cae6553da74969 (127.15.113.129:40037)
I20260809 22:53:56.652967 32290 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.113.129@KRBTEST.COM'} at 127.15.113.129:57709
WARNING: no policy specified for kudu/127.15.113.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.113.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.113.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.130.keytab.
Entry for principal kudu/127.15.113.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.130.keytab.
W20260809 22:53:56.709097 15814 mini_kdc.cc:262] Time spent creating service keytab for kudu/127.15.113.130: real 0.105s user 0.000s sys 0.045s
I20260809 22:53:56.710608 15814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskItYPJR/build/asan/bin/kudu
/tmp/dist-test-taskItYPJR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315935617919-15814-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315935617919-15814-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315935617919-15814-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315935617919-15814-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.113.130:0
--local_ip_for_outbound_sockets=127.15.113.130
--webserver_interface=127.15.113.130
--webserver_port=0
--tserver_master_addrs=127.15.113.190:44105
--keytab_file=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.130.keytab
--principal=kudu/127.15.113.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.113.148:46473
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315935617919-15814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315935617919-15814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315935617919-15814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:53:57.105396 32584 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:53:57.105664 32584 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:53:57.105805 32584 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:53:57.112732 32584 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:53:57.112851 32584 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:53:57.113044 32584 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.113.130
I20260809 22:53:57.122545 32584 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.113.148:46473
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315935617919-15814-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315935617919-15814-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.130.keytab
--principal=kudu/127.15.113.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.113.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315935617919-15814-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.113.130
--webserver_port=0
--tserver_master_addrs=127.15.113.190:44105
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.113.130
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315935617919-15814-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:53:57.123739 32584 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:53:57.125026 32584 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:53:57.137442 32591 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:53:57.137526 32592 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:53:57.137513 32597 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:53:57.138346 32584 server_base.cc:1034] running on GCE node
I20260809 22:53:57.138895 32584 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:53:57.139787 32584 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:53:57.141027 32584 hybrid_clock.cc:648] HybridClock initialized: now 1786316037140960 us; error 45 us; skew 500 ppm
I20260809 22:53:57.141356 32584 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:53:57 dist-test-slave-ncp9 krb5kdc[32227](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316037, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.113.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:53:57.144520 32584 init.cc:377] Logged in from keytab as kudu/127.15.113.130@KRBTEST.COM (short username kudu)
I20260809 22:53:57.145989 32584 webserver.cc:460] Webserver started at http://127.15.113.130:44087/ using document root <none> and password file <none>
I20260809 22:53:57.146674 32584 fs_manager.cc:359] Metadata directory not provided
I20260809 22:53:57.146799 32584 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:53:57.147095 32584 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:53:57.150105 32584 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315935617919-15814-0/minicluster-data/ts-1/data/instance:
uuid: "a08de66423a7459283a34dd914a0d4fc"
format_stamp: "Formatted at 2026-08-09 22:53:57 on dist-test-slave-ncp9"
server_key: "c302e61a3ecc65817da927fbc4072bb3"
server_key_iv: "b77b188edf50e23c6cb0afd6791dd8c1"
server_key_version: "encryptionkey@0"
I20260809 22:53:57.151014 32584 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315935617919-15814-0/minicluster-data/ts-1/wal/instance:
uuid: "a08de66423a7459283a34dd914a0d4fc"
format_stamp: "Formatted at 2026-08-09 22:53:57 on dist-test-slave-ncp9"
server_key: "c302e61a3ecc65817da927fbc4072bb3"
server_key_iv: "b77b188edf50e23c6cb0afd6791dd8c1"
server_key_version: "encryptionkey@0"
I20260809 22:53:57.156260 32584 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260809 22:53:57.159698 32609 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:53:57.161078 32584 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260809 22:53:57.161248 32584 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315935617919-15814-0/minicluster-data/ts-1/data,/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315935617919-15814-0/minicluster-data/ts-1/wal
uuid: "a08de66423a7459283a34dd914a0d4fc"
format_stamp: "Formatted at 2026-08-09 22:53:57 on dist-test-slave-ncp9"
server_key: "c302e61a3ecc65817da927fbc4072bb3"
server_key_iv: "b77b188edf50e23c6cb0afd6791dd8c1"
server_key_version: "encryptionkey@0"
I20260809 22:53:57.161424 32584 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315935617919-15814-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315935617919-15814-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315935617919-15814-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:53:57.180526 32584 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:53:57.184329 32584 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:53:57.184595 32584 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:53:57.185566 32584 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:53:57.186854 32584 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:53:57.186934 32584 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:53:57.187049 32584 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:53:57.187100 32584 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:53:57.219029 32584 rpc_server.cc:307] RPC server started. Bound to: 127.15.113.130:43457
I20260809 22:53:57.219062 32761 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.113.130:43457 every 8 connection(s)
I20260809 22:53:57.220238 32584 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315935617919-15814-0/minicluster-data/ts-1/data/info.pb
I20260809 22:53:57.225715 15814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskItYPJR/build/asan/bin/kudu as pid 32584
I20260809 22:53:57.225891 15814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315935617919-15814-0/minicluster-data/ts-1/wal/instance
I20260809 22:53:57.226438 15814 external_mini_cluster.cc:1428] Setting key e928cc3014e64fab57830dd1ee2d0199
Aug 09 22:53:57 dist-test-slave-ncp9 krb5kdc[32227](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316037, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.113.130@KRBTEST.COM for kudu/127.15.113.190@KRBTEST.COM
I20260809 22:53:57.241731 32764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:57.222824 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.113.130:58005 (local address 127.15.113.190:44105)
0809 22:53:57.223246 (+ 422us) server_negotiation.cc:206] Beginning negotiation
0809 22:53:57.223251 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:53:57.224287 (+ 1036us) server_negotiation.cc:407] Connection header received
0809 22:53:57.226497 (+ 2210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:53:57.226503 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:53:57.226589 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:53:57.226688 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:53:57.228439 (+ 1751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:57.229266 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:57.231522 (+ 2256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:57.231929 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:57.232017 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:57.236053 (+ 4036us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:53:57.236058 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:53:57.236061 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:53:57.236092 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:53:57.238692 (+ 2600us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:53:57.239563 (+ 871us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:53:57.239568 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:53:57.239570 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:53:57.239707 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:53:57.240132 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:53:57.240135 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:53:57.240137 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:53:57.240382 (+ 245us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:53:57.240496 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0809 22:53:57.241314 (+ 818us) server_negotiation.cc:299] Negotiation successful
0809 22:53:57.241479 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":317,"thread_start_us":209,"threads_started":1}
I20260809 22:53:57.243068 32767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:57.223407 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:44105 (local address 127.15.113.130:58005)
0809 22:53:57.224126 (+ 719us) negotiation.cc:107] Waiting for socket to connect
0809 22:53:57.224173 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0809 22:53:57.226184 (+ 2011us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:53:57.226972 (+ 788us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:53:57.226982 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:53:57.227511 (+ 529us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:53:57.228280 (+ 769us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:53:57.228301 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:53:57.229418 (+ 1117us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:53:57.229423 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:53:57.231393 (+ 1970us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:53:57.231409 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:53:57.232173 (+ 764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:53:57.232181 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:53:57.232458 (+ 277us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:57.233194 (+ 736us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:53:57.233227 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:53:57.235825 (+ 2598us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:53:57.238837 (+ 3012us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:53:57.238848 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:53:57.238862 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:53:57.239462 (+ 600us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:53:57.239819 (+ 357us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:53:57.239822 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:53:57.239825 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:53:57.240040 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:53:57.240503 (+ 463us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:53:57.240510 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:53:57.240927 (+ 417us) client_negotiation.cc:769] Sending connection context
0809 22:53:57.241258 (+ 331us) client_negotiation.cc:240] Negotiation successful
0809 22:53:57.241765 (+ 507us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":486,"thread_start_us":290,"threads_started":1}
I20260809 22:53:57.244433 32762 heartbeater.cc:347] Connected to a master server at 127.15.113.190:44105
I20260809 22:53:57.244762 32762 heartbeater.cc:464] Registering TS with master...
I20260809 22:53:57.245672 32762 heartbeater.cc:511] Master 127.15.113.190:44105 requested a full tablet report, sending...
I20260809 22:53:57.247161 32290 ts_manager.cc:194] Registered new tserver with Master: a08de66423a7459283a34dd914a0d4fc (127.15.113.130:43457)
I20260809 22:53:57.248111 32290 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.113.130@KRBTEST.COM'} at 127.15.113.130:58005
WARNING: no policy specified for kudu/127.15.113.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.113.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.113.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.131.keytab.
Entry for principal kudu/127.15.113.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.131.keytab.
I20260809 22:53:57.295970 15814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskItYPJR/build/asan/bin/kudu
/tmp/dist-test-taskItYPJR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315935617919-15814-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315935617919-15814-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315935617919-15814-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315935617919-15814-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.113.131:0
--local_ip_for_outbound_sockets=127.15.113.131
--webserver_interface=127.15.113.131
--webserver_port=0
--tserver_master_addrs=127.15.113.190:44105
--keytab_file=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.131.keytab
--principal=kudu/127.15.113.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.113.148:46473
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315935617919-15814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315935617919-15814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315935617919-15814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:53:57.623639 305 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:53:57.623909 305 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:53:57.624044 305 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:53:57.631148 305 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:53:57.631250 305 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:53:57.631403 305 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.113.131
I20260809 22:53:57.640879 305 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.113.148:46473
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315935617919-15814-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315935617919-15814-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.131.keytab
--principal=kudu/127.15.113.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.113.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315935617919-15814-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.113.131
--webserver_port=0
--tserver_master_addrs=127.15.113.190:44105
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.113.131
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315935617919-15814-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:53:57.642084 305 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:53:57.643386 305 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:53:57.655885 316 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:53:57.655941 315 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:53:57.655921 318 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:53:57.656749 305 server_base.cc:1034] running on GCE node
I20260809 22:53:57.657338 305 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:53:57.657711 32573 heartbeater.cc:503] Master 127.15.113.190:44105 was elected leader, sending a full tablet report...
I20260809 22:53:57.658277 305 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:53:57.659505 305 hybrid_clock.cc:648] HybridClock initialized: now 1786316037659446 us; error 51 us; skew 500 ppm
I20260809 22:53:57.659798 305 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:53:57 dist-test-slave-ncp9 krb5kdc[32227](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316037, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.113.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:53:57.662869 305 init.cc:377] Logged in from keytab as kudu/127.15.113.131@KRBTEST.COM (short username kudu)
I20260809 22:53:57.664168 305 webserver.cc:460] Webserver started at http://127.15.113.131:45365/ using document root <none> and password file <none>
I20260809 22:53:57.664861 305 fs_manager.cc:359] Metadata directory not provided
I20260809 22:53:57.665031 305 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:53:57.665328 305 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:53:57.668222 305 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315935617919-15814-0/minicluster-data/ts-2/data/instance:
uuid: "fa73d0285640440d9d39cf804139dfa6"
format_stamp: "Formatted at 2026-08-09 22:53:57 on dist-test-slave-ncp9"
server_key: "fb0bb05a3568dfc8c540e0c84a67e811"
server_key_iv: "1d07ec97304cefd114c2cd97f6ce45ae"
server_key_version: "encryptionkey@0"
I20260809 22:53:57.669016 305 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315935617919-15814-0/minicluster-data/ts-2/wal/instance:
uuid: "fa73d0285640440d9d39cf804139dfa6"
format_stamp: "Formatted at 2026-08-09 22:53:57 on dist-test-slave-ncp9"
server_key: "fb0bb05a3568dfc8c540e0c84a67e811"
server_key_iv: "1d07ec97304cefd114c2cd97f6ce45ae"
server_key_version: "encryptionkey@0"
I20260809 22:53:57.674245 305 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.003s
I20260809 22:53:57.677542 330 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:53:57.678834 305 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 22:53:57.678980 305 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315935617919-15814-0/minicluster-data/ts-2/data,/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315935617919-15814-0/minicluster-data/ts-2/wal
uuid: "fa73d0285640440d9d39cf804139dfa6"
format_stamp: "Formatted at 2026-08-09 22:53:57 on dist-test-slave-ncp9"
server_key: "fb0bb05a3568dfc8c540e0c84a67e811"
server_key_iv: "1d07ec97304cefd114c2cd97f6ce45ae"
server_key_version: "encryptionkey@0"
I20260809 22:53:57.679149 305 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315935617919-15814-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315935617919-15814-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315935617919-15814-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:53:57.694761 305 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:53:57.698313 305 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:53:57.698597 305 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:53:57.699592 305 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:53:57.701043 305 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:53:57.701120 305 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:53:57.701241 305 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:53:57.701299 305 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:53:57.732844 305 rpc_server.cc:307] RPC server started. Bound to: 127.15.113.131:33765
I20260809 22:53:57.732863 487 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.113.131:33765 every 8 connection(s)
I20260809 22:53:57.734113 305 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315935617919-15814-0/minicluster-data/ts-2/data/info.pb
I20260809 22:53:57.739482 15814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskItYPJR/build/asan/bin/kudu as pid 305
I20260809 22:53:57.739655 15814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315935617919-15814-0/minicluster-data/ts-2/wal/instance
I20260809 22:53:57.740193 15814 external_mini_cluster.cc:1428] Setting key d1219a701f42f5e2ef6acae2604dc23b
Aug 09 22:53:57 dist-test-slave-ncp9 krb5kdc[32227](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316037, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.113.131@KRBTEST.COM for kudu/127.15.113.190@KRBTEST.COM
I20260809 22:53:57.752663 32764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:57.736642 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.113.131:47879 (local address 127.15.113.190:44105)
0809 22:53:57.736826 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0809 22:53:57.736832 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:53:57.738025 (+ 1193us) server_negotiation.cc:407] Connection header received
0809 22:53:57.739868 (+ 1843us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:53:57.739875 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:53:57.739974 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:53:57.740087 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:53:57.741658 (+ 1571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:57.742639 (+ 981us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:57.744322 (+ 1683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:57.744761 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:57.744853 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:57.747996 (+ 3143us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:53:57.748001 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:53:57.748005 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:53:57.748035 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:53:57.750312 (+ 2277us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:53:57.750921 (+ 609us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:53:57.750924 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:53:57.750927 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:53:57.751036 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:53:57.751372 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:53:57.751375 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:53:57.751377 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:53:57.751618 (+ 241us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:53:57.751741 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0809 22:53:57.752252 (+ 511us) server_negotiation.cc:299] Negotiation successful
0809 22:53:57.752411 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260809 22:53:57.753543 491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:57.737158 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:44105 (local address 127.15.113.131:47879)
0809 22:53:57.737866 (+ 708us) negotiation.cc:107] Waiting for socket to connect
0809 22:53:57.737910 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 22:53:57.739569 (+ 1659us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:53:57.740268 (+ 699us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:53:57.740278 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:53:57.740859 (+ 581us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:53:57.741471 (+ 612us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:53:57.741496 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:53:57.742798 (+ 1302us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:53:57.742803 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:53:57.744191 (+ 1388us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:53:57.744203 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:53:57.744964 (+ 761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:53:57.744971 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:53:57.745171 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:57.745771 (+ 600us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:53:57.745792 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:53:57.747804 (+ 2012us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:53:57.750438 (+ 2634us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:53:57.750444 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:53:57.750454 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:53:57.750830 (+ 376us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:53:57.751108 (+ 278us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:53:57.751111 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:53:57.751113 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:53:57.751291 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:53:57.751715 (+ 424us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:53:57.751720 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:53:57.751982 (+ 262us) client_negotiation.cc:769] Sending connection context
0809 22:53:57.752190 (+ 208us) client_negotiation.cc:240] Negotiation successful
0809 22:53:57.752522 (+ 332us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":476,"thread_start_us":268,"threads_started":1}
I20260809 22:53:57.754760 488 heartbeater.cc:347] Connected to a master server at 127.15.113.190:44105
I20260809 22:53:57.755069 488 heartbeater.cc:464] Registering TS with master...
I20260809 22:53:57.755792 488 heartbeater.cc:511] Master 127.15.113.190:44105 requested a full tablet report, sending...
I20260809 22:53:57.757128 32290 ts_manager.cc:194] Registered new tserver with Master: fa73d0285640440d9d39cf804139dfa6 (127.15.113.131:33765)
I20260809 22:53:57.757859 32290 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.113.131@KRBTEST.COM'} at 127.15.113.131:47879
I20260809 22:53:57.767115 15814 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 22:53:57.783386 32764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:57.771411 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47996 (local address 127.15.113.190:44105)
0809 22:53:57.771562 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0809 22:53:57.771567 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:53:57.772022 (+ 455us) server_negotiation.cc:407] Connection header received
0809 22:53:57.772200 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:53:57.772203 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:53:57.772262 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:53:57.772342 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:53:57.773838 (+ 1496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:57.774522 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:57.775775 (+ 1253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:57.776185 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:57.776292 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:57.778498 (+ 2206us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:53:57.778503 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:53:57.778506 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:53:57.778536 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:53:57.780737 (+ 2201us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:53:57.781414 (+ 677us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:53:57.781420 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:53:57.781423 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:53:57.781550 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:53:57.781932 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:53:57.781936 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:53:57.781938 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:53:57.782261 (+ 323us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:53:57.782508 (+ 247us) server_negotiation.cc:1035] Waiting for connection context
0809 22:53:57.782878 (+ 370us) server_negotiation.cc:299] Negotiation successful
0809 22:53:57.783083 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260809 22:53:57.787050 32290 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47996:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 22:53:57.793792 32290 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 22:53:57.816799 513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:57.806683 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54922 (local address 127.15.113.130:43457)
0809 22:53:57.808991 (+ 2308us) server_negotiation.cc:206] Beginning negotiation
0809 22:53:57.808999 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 22:53:57.809025 (+ 26us) server_negotiation.cc:407] Connection header received
0809 22:53:57.809116 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:53:57.809123 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:53:57.809380 (+ 257us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:53:57.809574 (+ 194us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:53:57.810775 (+ 1201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:57.811990 (+ 1215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:57.814501 (+ 2511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:57.815637 (+ 1136us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:57.815806 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:57.815977 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0809 22:53:57.816324 (+ 347us) server_negotiation.cc:299] Negotiation successful
0809 22:53:57.816491 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2133,"thread_start_us":2025,"threads_started":1}
I20260809 22:53:57.816854 512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:57.806167 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.130:43457 (local address 127.0.0.1:54922)
0809 22:53:57.808396 (+ 2229us) negotiation.cc:107] Waiting for socket to connect
0809 22:53:57.808412 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 22:53:57.808517 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:53:57.809726 (+ 1209us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:53:57.809730 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:53:57.809749 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:53:57.810604 (+ 855us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:53:57.810616 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:53:57.812158 (+ 1542us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:53:57.812163 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:53:57.814300 (+ 2137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:53:57.814315 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:53:57.815858 (+ 1543us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:53:57.815866 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:53:57.816060 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:57.816111 (+ 51us) client_negotiation.cc:769] Sending connection context
0809 22:53:57.816253 (+ 142us) client_negotiation.cc:240] Negotiation successful
0809 22:53:57.816430 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2119,"mutex_wait_us":505,"thread_start_us":2002,"threads_started":1}
I20260809 22:53:57.818063 514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:57.808494 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.131:33765 (local address 127.0.0.1:42544)
0809 22:53:57.809393 (+ 899us) negotiation.cc:107] Waiting for socket to connect
0809 22:53:57.809408 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 22:53:57.809510 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:53:57.810779 (+ 1269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:53:57.810784 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:53:57.810809 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:53:57.811158 (+ 349us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:53:57.811170 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:53:57.813314 (+ 2144us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:53:57.813319 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:53:57.815353 (+ 2034us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:53:57.815364 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:53:57.817270 (+ 1906us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:53:57.817277 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:53:57.817438 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:57.817471 (+ 33us) client_negotiation.cc:769] Sending connection context
0809 22:53:57.817552 (+ 81us) client_negotiation.cc:240] Negotiation successful
0809 22:53:57.817679 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":789,"mutex_wait_us":2,"thread_start_us":659,"threads_started":1}
I20260809 22:53:57.818082 515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:57.809006 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42544 (local address 127.15.113.131:33765)
0809 22:53:57.810226 (+ 1220us) server_negotiation.cc:206] Beginning negotiation
0809 22:53:57.810235 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 22:53:57.810265 (+ 30us) server_negotiation.cc:407] Connection header received
0809 22:53:57.810356 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:53:57.810363 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:53:57.810587 (+ 224us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:53:57.810780 (+ 193us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:53:57.811364 (+ 584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:57.813139 (+ 1775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:57.815495 (+ 2356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:57.816759 (+ 1264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:57.816958 (+ 199us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:57.817108 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0809 22:53:57.817629 (+ 521us) server_negotiation.cc:299] Negotiation successful
0809 22:53:57.817794 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1056,"thread_start_us":895,"threads_started":1}
I20260809 22:53:57.818470 509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:57.805840 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.129:40037 (local address 127.0.0.1:51734)
0809 22:53:57.807626 (+ 1786us) negotiation.cc:107] Waiting for socket to connect
0809 22:53:57.807658 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0809 22:53:57.807851 (+ 193us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:53:57.809697 (+ 1846us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:53:57.809705 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:53:57.809734 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:53:57.810150 (+ 416us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:53:57.810164 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:53:57.812678 (+ 2514us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:53:57.812683 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:53:57.814964 (+ 2281us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:53:57.814980 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:53:57.817700 (+ 2720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:53:57.817706 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:53:57.817876 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:57.817907 (+ 31us) client_negotiation.cc:769] Sending connection context
0809 22:53:57.817976 (+ 69us) client_negotiation.cc:240] Negotiation successful
0809 22:53:57.818099 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1576,"mutex_wait_us":3,"thread_start_us":1464,"threads_started":1}
I20260809 22:53:57.819547 510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:57.806103 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51734 (local address 127.15.113.129:40037)
0809 22:53:57.806876 (+ 773us) server_negotiation.cc:206] Beginning negotiation
0809 22:53:57.806885 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 22:53:57.807783 (+ 898us) server_negotiation.cc:407] Connection header received
0809 22:53:57.808121 (+ 338us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:53:57.808128 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:53:57.808413 (+ 285us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:53:57.810813 (+ 2400us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:53:57.811074 (+ 261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:57.812526 (+ 1452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:57.815144 (+ 2618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:57.817458 (+ 2314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:57.818857 (+ 1399us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:57.819013 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0809 22:53:57.819078 (+ 65us) server_negotiation.cc:299] Negotiation successful
0809 22:53:57.819245 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":608,"thread_start_us":437,"threads_started":1}
I20260809 22:53:57.821249 32673 tablet_service.cc:1467] Processing CreateTablet for tablet 9c6218d3337e48258d528bc4112f7f79 (DEFAULT_TABLE table=test-table [id=38992647063a4820b72fa1c960dad4cf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:53:57.822793 32673 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9c6218d3337e48258d528bc4112f7f79. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:53:57.822727 32479 tablet_service.cc:1467] Processing CreateTablet for tablet 9c6218d3337e48258d528bc4112f7f79 (DEFAULT_TABLE table=test-table [id=38992647063a4820b72fa1c960dad4cf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:53:57.823210 392 tablet_service.cc:1467] Processing CreateTablet for tablet 9c6218d3337e48258d528bc4112f7f79 (DEFAULT_TABLE table=test-table [id=38992647063a4820b72fa1c960dad4cf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:53:57.823937 32479 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9c6218d3337e48258d528bc4112f7f79. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:53:57.824599 392 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9c6218d3337e48258d528bc4112f7f79. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:53:57.834744 519 tablet_bootstrap.cc:492] T 9c6218d3337e48258d528bc4112f7f79 P 6d150f09be4a4c43b2cae6553da74969: Bootstrap starting.
I20260809 22:53:57.836938 520 tablet_bootstrap.cc:492] T 9c6218d3337e48258d528bc4112f7f79 P a08de66423a7459283a34dd914a0d4fc: Bootstrap starting.
I20260809 22:53:57.838529 519 tablet_bootstrap.cc:654] T 9c6218d3337e48258d528bc4112f7f79 P 6d150f09be4a4c43b2cae6553da74969: Neither blocks nor log segments found. Creating new log.
I20260809 22:53:57.839378 521 tablet_bootstrap.cc:492] T 9c6218d3337e48258d528bc4112f7f79 P fa73d0285640440d9d39cf804139dfa6: Bootstrap starting.
I20260809 22:53:57.840029 519 log.cc:824] T 9c6218d3337e48258d528bc4112f7f79 P 6d150f09be4a4c43b2cae6553da74969: Log is configured to *not* fsync() on all Append() calls
I20260809 22:53:57.840296 520 tablet_bootstrap.cc:654] T 9c6218d3337e48258d528bc4112f7f79 P a08de66423a7459283a34dd914a0d4fc: Neither blocks nor log segments found. Creating new log.
I20260809 22:53:57.841356 520 log.cc:824] T 9c6218d3337e48258d528bc4112f7f79 P a08de66423a7459283a34dd914a0d4fc: Log is configured to *not* fsync() on all Append() calls
I20260809 22:53:57.842978 521 tablet_bootstrap.cc:654] T 9c6218d3337e48258d528bc4112f7f79 P fa73d0285640440d9d39cf804139dfa6: Neither blocks nor log segments found. Creating new log.
I20260809 22:53:57.843277 519 tablet_bootstrap.cc:492] T 9c6218d3337e48258d528bc4112f7f79 P 6d150f09be4a4c43b2cae6553da74969: No bootstrap required, opened a new log
I20260809 22:53:57.843557 519 ts_tablet_manager.cc:1397] T 9c6218d3337e48258d528bc4112f7f79 P 6d150f09be4a4c43b2cae6553da74969: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260809 22:53:57.843926 521 log.cc:824] T 9c6218d3337e48258d528bc4112f7f79 P fa73d0285640440d9d39cf804139dfa6: Log is configured to *not* fsync() on all Append() calls
I20260809 22:53:57.844445 520 tablet_bootstrap.cc:492] T 9c6218d3337e48258d528bc4112f7f79 P a08de66423a7459283a34dd914a0d4fc: No bootstrap required, opened a new log
I20260809 22:53:57.844705 520 ts_tablet_manager.cc:1397] T 9c6218d3337e48258d528bc4112f7f79 P a08de66423a7459283a34dd914a0d4fc: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260809 22:53:57.846314 521 tablet_bootstrap.cc:492] T 9c6218d3337e48258d528bc4112f7f79 P fa73d0285640440d9d39cf804139dfa6: No bootstrap required, opened a new log
I20260809 22:53:57.846604 521 ts_tablet_manager.cc:1397] T 9c6218d3337e48258d528bc4112f7f79 P fa73d0285640440d9d39cf804139dfa6: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.007s
I20260809 22:53:57.853868 519 raft_consensus.cc:348] T 9c6218d3337e48258d528bc4112f7f79 P 6d150f09be4a4c43b2cae6553da74969 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d150f09be4a4c43b2cae6553da74969" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 40037 } } peers { permanent_uuid: "a08de66423a7459283a34dd914a0d4fc" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 43457 } } peers { permanent_uuid: "fa73d0285640440d9d39cf804139dfa6" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 33765 } }
I20260809 22:53:57.854291 519 raft_consensus.cc:374] T 9c6218d3337e48258d528bc4112f7f79 P 6d150f09be4a4c43b2cae6553da74969 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:53:57.854128 521 raft_consensus.cc:348] T 9c6218d3337e48258d528bc4112f7f79 P fa73d0285640440d9d39cf804139dfa6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d150f09be4a4c43b2cae6553da74969" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 40037 } } peers { permanent_uuid: "a08de66423a7459283a34dd914a0d4fc" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 43457 } } peers { permanent_uuid: "fa73d0285640440d9d39cf804139dfa6" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 33765 } }
I20260809 22:53:57.854399 519 raft_consensus.cc:729] T 9c6218d3337e48258d528bc4112f7f79 P 6d150f09be4a4c43b2cae6553da74969 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6d150f09be4a4c43b2cae6553da74969, State: Initialized, Role: FOLLOWER
I20260809 22:53:57.854489 521 raft_consensus.cc:374] T 9c6218d3337e48258d528bc4112f7f79 P fa73d0285640440d9d39cf804139dfa6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:53:57.854586 521 raft_consensus.cc:729] T 9c6218d3337e48258d528bc4112f7f79 P fa73d0285640440d9d39cf804139dfa6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa73d0285640440d9d39cf804139dfa6, State: Initialized, Role: FOLLOWER
I20260809 22:53:57.854626 520 raft_consensus.cc:348] T 9c6218d3337e48258d528bc4112f7f79 P a08de66423a7459283a34dd914a0d4fc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d150f09be4a4c43b2cae6553da74969" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 40037 } } peers { permanent_uuid: "a08de66423a7459283a34dd914a0d4fc" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 43457 } } peers { permanent_uuid: "fa73d0285640440d9d39cf804139dfa6" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 33765 } }
I20260809 22:53:57.855029 520 raft_consensus.cc:374] T 9c6218d3337e48258d528bc4112f7f79 P a08de66423a7459283a34dd914a0d4fc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:53:57.855145 520 raft_consensus.cc:729] T 9c6218d3337e48258d528bc4112f7f79 P a08de66423a7459283a34dd914a0d4fc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a08de66423a7459283a34dd914a0d4fc, State: Initialized, Role: FOLLOWER
I20260809 22:53:57.855029 519 consensus_queue.cc:260] T 9c6218d3337e48258d528bc4112f7f79 P 6d150f09be4a4c43b2cae6553da74969 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d150f09be4a4c43b2cae6553da74969" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 40037 } } peers { permanent_uuid: "a08de66423a7459283a34dd914a0d4fc" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 43457 } } peers { permanent_uuid: "fa73d0285640440d9d39cf804139dfa6" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 33765 } }
I20260809 22:53:57.855154 521 consensus_queue.cc:260] T 9c6218d3337e48258d528bc4112f7f79 P fa73d0285640440d9d39cf804139dfa6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d150f09be4a4c43b2cae6553da74969" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 40037 } } peers { permanent_uuid: "a08de66423a7459283a34dd914a0d4fc" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 43457 } } peers { permanent_uuid: "fa73d0285640440d9d39cf804139dfa6" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 33765 } }
I20260809 22:53:57.855787 520 consensus_queue.cc:260] T 9c6218d3337e48258d528bc4112f7f79 P a08de66423a7459283a34dd914a0d4fc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d150f09be4a4c43b2cae6553da74969" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 40037 } } peers { permanent_uuid: "a08de66423a7459283a34dd914a0d4fc" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 43457 } } peers { permanent_uuid: "fa73d0285640440d9d39cf804139dfa6" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 33765 } }
I20260809 22:53:57.856426 488 heartbeater.cc:503] Master 127.15.113.190:44105 was elected leader, sending a full tablet report...
I20260809 22:53:57.856941 521 ts_tablet_manager.cc:1428] T 9c6218d3337e48258d528bc4112f7f79 P fa73d0285640440d9d39cf804139dfa6: Time spent starting tablet: real 0.010s user 0.010s sys 0.000s
I20260809 22:53:57.861567 32762 heartbeater.cc:503] Master 127.15.113.190:44105 was elected leader, sending a full tablet report...
I20260809 22:53:57.862283 520 ts_tablet_manager.cc:1428] T 9c6218d3337e48258d528bc4112f7f79 P a08de66423a7459283a34dd914a0d4fc: Time spent starting tablet: real 0.017s user 0.008s sys 0.010s
I20260809 22:53:57.863834 519 ts_tablet_manager.cc:1428] T 9c6218d3337e48258d528bc4112f7f79 P 6d150f09be4a4c43b2cae6553da74969: Time spent starting tablet: real 0.020s user 0.021s sys 0.000s
W20260809 22:53:57.972275 32763 tablet.cc:2369] T 9c6218d3337e48258d528bc4112f7f79 P a08de66423a7459283a34dd914a0d4fc: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 22:53:57.985637 489 tablet.cc:2369] T 9c6218d3337e48258d528bc4112f7f79 P fa73d0285640440d9d39cf804139dfa6: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 22:53:58.051903 527 raft_consensus.cc:482] T 9c6218d3337e48258d528bc4112f7f79 P 6d150f09be4a4c43b2cae6553da74969 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 22:53:58.052187 527 raft_consensus.cc:504] T 9c6218d3337e48258d528bc4112f7f79 P 6d150f09be4a4c43b2cae6553da74969 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d150f09be4a4c43b2cae6553da74969" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 40037 } } peers { permanent_uuid: "a08de66423a7459283a34dd914a0d4fc" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 43457 } } peers { permanent_uuid: "fa73d0285640440d9d39cf804139dfa6" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 33765 } }
I20260809 22:53:58.053910 527 leader_election.cc:288] T 9c6218d3337e48258d528bc4112f7f79 P 6d150f09be4a4c43b2cae6553da74969 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a08de66423a7459283a34dd914a0d4fc (127.15.113.130:43457), fa73d0285640440d9d39cf804139dfa6 (127.15.113.131:33765)
I20260809 22:53:58.060997 515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:58.055074 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.113.129:49567 (local address 127.15.113.131:33765)
0809 22:53:58.055266 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0809 22:53:58.055272 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:53:58.055673 (+ 401us) server_negotiation.cc:407] Connection header received
0809 22:53:58.055883 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:53:58.055888 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:53:58.055965 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:53:58.056048 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:53:58.056654 (+ 606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:58.057497 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:58.059246 (+ 1749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:58.060001 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:58.060157 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:58.060229 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0809 22:53:58.060589 (+ 360us) server_negotiation.cc:299] Negotiation successful
0809 22:53:58.060723 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260809 22:53:58.061015 513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:58.054425 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.113.129:49191 (local address 127.15.113.130:43457)
0809 22:53:58.054687 (+ 262us) server_negotiation.cc:206] Beginning negotiation
0809 22:53:58.054694 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:53:58.054871 (+ 177us) server_negotiation.cc:407] Connection header received
0809 22:53:58.055112 (+ 241us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:53:58.055118 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:53:58.055211 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:53:58.055279 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:53:58.056082 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:58.057207 (+ 1125us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:58.059094 (+ 1887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:58.059926 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:58.060061 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:58.060133 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0809 22:53:58.060603 (+ 470us) server_negotiation.cc:299] Negotiation successful
0809 22:53:58.060727 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260809 22:53:58.061280 532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:58.054841 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.131:33765 (local address 127.15.113.129:49567)
0809 22:53:58.055531 (+ 690us) negotiation.cc:107] Waiting for socket to connect
0809 22:53:58.055547 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 22:53:58.055728 (+ 181us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:53:58.056082 (+ 354us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:53:58.056087 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:53:58.056115 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:53:58.056465 (+ 350us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:53:58.056477 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:53:58.057647 (+ 1170us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:53:58.057651 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:53:58.059087 (+ 1436us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:53:58.059102 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:53:58.060230 (+ 1128us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:53:58.060235 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:53:58.060399 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:58.060436 (+ 37us) client_negotiation.cc:769] Sending connection context
0809 22:53:58.060547 (+ 111us) client_negotiation.cc:240] Negotiation successful
0809 22:53:58.060673 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":532,"mutex_wait_us":6,"thread_start_us":266,"threads_started":1}
I20260809 22:53:58.061378 531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:58.054117 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.130:43457 (local address 127.15.113.129:49191)
0809 22:53:58.054732 (+ 615us) negotiation.cc:107] Waiting for socket to connect
0809 22:53:58.054752 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 22:53:58.054922 (+ 170us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:53:58.055472 (+ 550us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:53:58.055475 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:53:58.055500 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:53:58.055887 (+ 387us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:53:58.055901 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:53:58.057339 (+ 1438us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:53:58.057345 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:53:58.058915 (+ 1570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:53:58.058928 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:53:58.060152 (+ 1224us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:53:58.060159 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:53:58.060401 (+ 242us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:58.060446 (+ 45us) client_negotiation.cc:769] Sending connection context
0809 22:53:58.060529 (+ 83us) client_negotiation.cc:240] Negotiation successful
0809 22:53:58.060677 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":465,"mutex_wait_us":71,"thread_start_us":289,"threads_started":1}
I20260809 22:53:58.062273 424 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9c6218d3337e48258d528bc4112f7f79" candidate_uuid: "6d150f09be4a4c43b2cae6553da74969" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa73d0285640440d9d39cf804139dfa6" is_pre_election: true
I20260809 22:53:58.062290 32698 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9c6218d3337e48258d528bc4112f7f79" candidate_uuid: "6d150f09be4a4c43b2cae6553da74969" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a08de66423a7459283a34dd914a0d4fc" is_pre_election: true
I20260809 22:53:58.062752 32698 raft_consensus.cc:2454] T 9c6218d3337e48258d528bc4112f7f79 P a08de66423a7459283a34dd914a0d4fc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6d150f09be4a4c43b2cae6553da74969 in term 0.
I20260809 22:53:58.062752 424 raft_consensus.cc:2454] T 9c6218d3337e48258d528bc4112f7f79 P fa73d0285640440d9d39cf804139dfa6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6d150f09be4a4c43b2cae6553da74969 in term 0.
I20260809 22:53:58.063369 32417 leader_election.cc:302] T 9c6218d3337e48258d528bc4112f7f79 P 6d150f09be4a4c43b2cae6553da74969 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6d150f09be4a4c43b2cae6553da74969, a08de66423a7459283a34dd914a0d4fc; no voters:
I20260809 22:53:58.063710 527 raft_consensus.cc:2781] T 9c6218d3337e48258d528bc4112f7f79 P 6d150f09be4a4c43b2cae6553da74969 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 22:53:58.063868 527 raft_consensus.cc:482] T 9c6218d3337e48258d528bc4112f7f79 P 6d150f09be4a4c43b2cae6553da74969 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 22:53:58.063977 527 raft_consensus.cc:3037] T 9c6218d3337e48258d528bc4112f7f79 P 6d150f09be4a4c43b2cae6553da74969 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:53:58.066361 527 raft_consensus.cc:504] T 9c6218d3337e48258d528bc4112f7f79 P 6d150f09be4a4c43b2cae6553da74969 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d150f09be4a4c43b2cae6553da74969" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 40037 } } peers { permanent_uuid: "a08de66423a7459283a34dd914a0d4fc" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 43457 } } peers { permanent_uuid: "fa73d0285640440d9d39cf804139dfa6" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 33765 } }
I20260809 22:53:58.066916 527 leader_election.cc:288] T 9c6218d3337e48258d528bc4112f7f79 P 6d150f09be4a4c43b2cae6553da74969 [CANDIDATE]: Term 1 election: Requested vote from peers a08de66423a7459283a34dd914a0d4fc (127.15.113.130:43457), fa73d0285640440d9d39cf804139dfa6 (127.15.113.131:33765)
I20260809 22:53:58.067337 32698 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9c6218d3337e48258d528bc4112f7f79" candidate_uuid: "6d150f09be4a4c43b2cae6553da74969" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a08de66423a7459283a34dd914a0d4fc"
I20260809 22:53:58.067415 424 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9c6218d3337e48258d528bc4112f7f79" candidate_uuid: "6d150f09be4a4c43b2cae6553da74969" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa73d0285640440d9d39cf804139dfa6"
I20260809 22:53:58.067507 32698 raft_consensus.cc:3037] T 9c6218d3337e48258d528bc4112f7f79 P a08de66423a7459283a34dd914a0d4fc [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:53:58.067605 424 raft_consensus.cc:3037] T 9c6218d3337e48258d528bc4112f7f79 P fa73d0285640440d9d39cf804139dfa6 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:53:58.069833 424 raft_consensus.cc:2454] T 9c6218d3337e48258d528bc4112f7f79 P fa73d0285640440d9d39cf804139dfa6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6d150f09be4a4c43b2cae6553da74969 in term 1.
I20260809 22:53:58.069881 32698 raft_consensus.cc:2454] T 9c6218d3337e48258d528bc4112f7f79 P a08de66423a7459283a34dd914a0d4fc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6d150f09be4a4c43b2cae6553da74969 in term 1.
I20260809 22:53:58.070199 32417 leader_election.cc:302] T 9c6218d3337e48258d528bc4112f7f79 P 6d150f09be4a4c43b2cae6553da74969 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6d150f09be4a4c43b2cae6553da74969, fa73d0285640440d9d39cf804139dfa6; no voters:
I20260809 22:53:58.070463 527 raft_consensus.cc:2781] T 9c6218d3337e48258d528bc4112f7f79 P 6d150f09be4a4c43b2cae6553da74969 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 22:53:58.070880 527 raft_consensus.cc:686] T 9c6218d3337e48258d528bc4112f7f79 P 6d150f09be4a4c43b2cae6553da74969 [term 1 LEADER]: Becoming Leader. State: Replica: 6d150f09be4a4c43b2cae6553da74969, State: Running, Role: LEADER
I20260809 22:53:58.071305 527 consensus_queue.cc:237] T 9c6218d3337e48258d528bc4112f7f79 P 6d150f09be4a4c43b2cae6553da74969 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d150f09be4a4c43b2cae6553da74969" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 40037 } } peers { permanent_uuid: "a08de66423a7459283a34dd914a0d4fc" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 43457 } } peers { permanent_uuid: "fa73d0285640440d9d39cf804139dfa6" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 33765 } }
I20260809 22:53:58.075634 32289 catalog_manager.cc:5495] T 9c6218d3337e48258d528bc4112f7f79 P 6d150f09be4a4c43b2cae6553da74969 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6d150f09be4a4c43b2cae6553da74969 (127.15.113.129). New cstate: current_term: 1 leader_uuid: "6d150f09be4a4c43b2cae6553da74969" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d150f09be4a4c43b2cae6553da74969" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 40037 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a08de66423a7459283a34dd914a0d4fc" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 43457 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fa73d0285640440d9d39cf804139dfa6" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 33765 } health_report { overall_health: UNKNOWN } } }
W20260809 22:53:58.097599 32574 tablet.cc:2369] T 9c6218d3337e48258d528bc4112f7f79 P 6d150f09be4a4c43b2cae6553da74969: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 22:53:58.110250 510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:58.105477 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51740 (local address 127.15.113.129:40037)
0809 22:53:58.105701 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0809 22:53:58.105706 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:53:58.105724 (+ 18us) server_negotiation.cc:407] Connection header received
0809 22:53:58.105914 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:53:58.105918 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:53:58.106010 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:53:58.106140 (+ 130us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 22:53:58.106435 (+ 295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:58.107138 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:58.108358 (+ 1220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:58.108739 (+ 381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:58.108822 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:58.109287 (+ 465us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 22:53:58.109769 (+ 482us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 22:53:58.109875 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0809 22:53:58.109929 (+ 54us) server_negotiation.cc:299] Negotiation successful
0809 22:53:58.109993 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260809 22:53:58.118387 32698 raft_consensus.cc:1261] T 9c6218d3337e48258d528bc4112f7f79 P a08de66423a7459283a34dd914a0d4fc [term 1 FOLLOWER]: Refusing update from remote peer 6d150f09be4a4c43b2cae6553da74969: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 22:53:58.118402 424 raft_consensus.cc:1261] T 9c6218d3337e48258d528bc4112f7f79 P fa73d0285640440d9d39cf804139dfa6 [term 1 FOLLOWER]: Refusing update from remote peer 6d150f09be4a4c43b2cae6553da74969: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 22:53:58.119302 533 consensus_queue.cc:1035] T 9c6218d3337e48258d528bc4112f7f79 P 6d150f09be4a4c43b2cae6553da74969 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fa73d0285640440d9d39cf804139dfa6" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 33765 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 22:53:58.119671 527 consensus_queue.cc:1035] T 9c6218d3337e48258d528bc4112f7f79 P 6d150f09be4a4c43b2cae6553da74969 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a08de66423a7459283a34dd914a0d4fc" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 43457 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 22:53:58.143087 32289 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47996:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 22:53:58.143411 32289 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47996:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 22:53:58.146942 32289 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ac5bc82598e64cdd856390462e434519: Sending DeleteTablet for 3 replicas of tablet 9c6218d3337e48258d528bc4112f7f79
I20260809 22:53:58.148018 32479 tablet_service.cc:1514] Processing DeleteTablet for tablet 9c6218d3337e48258d528bc4112f7f79 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:53:58 UTC) from {username='kudu', principal='kudu/127.15.113.190@KRBTEST.COM'} at 127.0.0.1:51734
I20260809 22:53:58.148166 32673 tablet_service.cc:1514] Processing DeleteTablet for tablet 9c6218d3337e48258d528bc4112f7f79 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:53:58 UTC) from {username='kudu', principal='kudu/127.15.113.190@KRBTEST.COM'} at 127.0.0.1:54922
I20260809 22:53:58.148705 392 tablet_service.cc:1514] Processing DeleteTablet for tablet 9c6218d3337e48258d528bc4112f7f79 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:53:58 UTC) from {username='kudu', principal='kudu/127.15.113.190@KRBTEST.COM'} at 127.0.0.1:42544
I20260809 22:53:58.149191 553 tablet_replica.cc:330] T 9c6218d3337e48258d528bc4112f7f79 P a08de66423a7459283a34dd914a0d4fc: stopping tablet replica
I20260809 22:53:58.149657 553 raft_consensus.cc:2229] T 9c6218d3337e48258d528bc4112f7f79 P a08de66423a7459283a34dd914a0d4fc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 22:53:58.149773 552 tablet_replica.cc:330] T 9c6218d3337e48258d528bc4112f7f79 P 6d150f09be4a4c43b2cae6553da74969: stopping tablet replica
I20260809 22:53:58.149868 15814 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskItYPJR/build/asan/bin/kudu pid 32390
I20260809 22:53:58.150177 552 raft_consensus.cc:2229] T 9c6218d3337e48258d528bc4112f7f79 P 6d150f09be4a4c43b2cae6553da74969 [term 1 LEADER]: Raft consensus shutting down.
I20260809 22:53:58.150350 553 raft_consensus.cc:2258] T 9c6218d3337e48258d528bc4112f7f79 P a08de66423a7459283a34dd914a0d4fc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 22:53:58.150801 552 raft_consensus.cc:2258] T 9c6218d3337e48258d528bc4112f7f79 P 6d150f09be4a4c43b2cae6553da74969 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 22:53:58.151490 554 tablet_replica.cc:330] T 9c6218d3337e48258d528bc4112f7f79 P fa73d0285640440d9d39cf804139dfa6: stopping tablet replica
I20260809 22:53:58.151854 554 raft_consensus.cc:2229] T 9c6218d3337e48258d528bc4112f7f79 P fa73d0285640440d9d39cf804139dfa6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 22:53:58.152328 554 raft_consensus.cc:2258] T 9c6218d3337e48258d528bc4112f7f79 P fa73d0285640440d9d39cf804139dfa6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 22:53:58.154585 554 ts_tablet_manager.cc:1905] T 9c6218d3337e48258d528bc4112f7f79 P fa73d0285640440d9d39cf804139dfa6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 22:53:58.154767 552 ts_tablet_manager.cc:1905] T 9c6218d3337e48258d528bc4112f7f79 P 6d150f09be4a4c43b2cae6553da74969: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 22:53:58.162142 554 ts_tablet_manager.cc:1918] T 9c6218d3337e48258d528bc4112f7f79 P fa73d0285640440d9d39cf804139dfa6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 22:53:58.162266 552 ts_tablet_manager.cc:1918] T 9c6218d3337e48258d528bc4112f7f79 P 6d150f09be4a4c43b2cae6553da74969: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 22:53:58.162321 554 log.cc:1196] T 9c6218d3337e48258d528bc4112f7f79 P fa73d0285640440d9d39cf804139dfa6: Deleting WAL directory at /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315935617919-15814-0/minicluster-data/ts-2/wal/wals/9c6218d3337e48258d528bc4112f7f79
I20260809 22:53:58.162437 552 log.cc:1196] T 9c6218d3337e48258d528bc4112f7f79 P 6d150f09be4a4c43b2cae6553da74969: Deleting WAL directory at /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315935617919-15814-0/minicluster-data/ts-0/wal/wals/9c6218d3337e48258d528bc4112f7f79
I20260809 22:53:58.162910 554 ts_tablet_manager.cc:1939] T 9c6218d3337e48258d528bc4112f7f79 P fa73d0285640440d9d39cf804139dfa6: Deleting consensus metadata
I20260809 22:53:58.162978 552 ts_tablet_manager.cc:1939] T 9c6218d3337e48258d528bc4112f7f79 P 6d150f09be4a4c43b2cae6553da74969: Deleting consensus metadata
I20260809 22:53:58.164738 32263 catalog_manager.cc:4841] TS 6d150f09be4a4c43b2cae6553da74969 (127.15.113.129:40037): tablet 9c6218d3337e48258d528bc4112f7f79 (table test-table [id=38992647063a4820b72fa1c960dad4cf]) successfully deleted
I20260809 22:53:58.164873 32268 catalog_manager.cc:4841] TS fa73d0285640440d9d39cf804139dfa6 (127.15.113.131:33765): tablet 9c6218d3337e48258d528bc4112f7f79 (table test-table [id=38992647063a4820b72fa1c960dad4cf]) successfully deleted
I20260809 22:53:58.165880 553 ts_tablet_manager.cc:1905] T 9c6218d3337e48258d528bc4112f7f79 P a08de66423a7459283a34dd914a0d4fc: Deleting tablet data with delete state TABLET_DATA_DELETED
Aug 09 22:53:58 dist-test-slave-ncp9 krb5kdc[32227](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316034, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.113.129@KRBTEST.COM
I20260809 22:53:58.172792 553 ts_tablet_manager.cc:1918] T 9c6218d3337e48258d528bc4112f7f79 P a08de66423a7459283a34dd914a0d4fc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 22:53:58.172986 553 log.cc:1196] T 9c6218d3337e48258d528bc4112f7f79 P a08de66423a7459283a34dd914a0d4fc: Deleting WAL directory at /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315935617919-15814-0/minicluster-data/ts-1/wal/wals/9c6218d3337e48258d528bc4112f7f79
I20260809 22:53:58.173463 510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:58.151449 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51750 (local address 127.15.113.129:40037)
0809 22:53:58.151617 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0809 22:53:58.151622 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:53:58.156637 (+ 5015us) server_negotiation.cc:407] Connection header received
0809 22:53:58.156827 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:53:58.156832 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:53:58.156936 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:53:58.157043 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:53:58.158674 (+ 1631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:58.159905 (+ 1231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:58.161742 (+ 1837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:58.162506 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:58.162608 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:58.167467 (+ 4859us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:53:58.167479 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:53:58.167493 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:53:58.167526 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:53:58.170394 (+ 2868us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:53:58.171059 (+ 665us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:53:58.171066 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:53:58.171068 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:53:58.171197 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:53:58.171590 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:53:58.171593 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:53:58.171595 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:53:58.171937 (+ 342us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:53:58.172230 (+ 293us) server_negotiation.cc:1035] Waiting for connection context
0809 22:53:58.172531 (+ 301us) server_negotiation.cc:299] Negotiation successful
0809 22:53:58.172768 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260809 22:53:58.173523 553 ts_tablet_manager.cc:1939] T 9c6218d3337e48258d528bc4112f7f79 P a08de66423a7459283a34dd914a0d4fc: Deleting consensus metadata
I20260809 22:53:58.174095 32567 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:53:58.175139 32268 catalog_manager.cc:4841] TS a08de66423a7459283a34dd914a0d4fc (127.15.113.130:43457): tablet 9c6218d3337e48258d528bc4112f7f79 (table test-table [id=38992647063a4820b72fa1c960dad4cf]) successfully deleted
I20260809 22:53:58.270203 15814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskItYPJR/build/asan/bin/kudu with pid 32390
I20260809 22:53:58.294689 15814 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskItYPJR/build/asan/bin/kudu pid 32584
Aug 09 22:53:58 dist-test-slave-ncp9 krb5kdc[32227](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316034, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.113.130@KRBTEST.COM
I20260809 22:53:58.309186 513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:58.295445 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54924 (local address 127.15.113.130:43457)
0809 22:53:58.295650 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0809 22:53:58.295657 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:53:58.295678 (+ 21us) server_negotiation.cc:407] Connection header received
0809 22:53:58.295781 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:53:58.295787 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:53:58.295865 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:53:58.295951 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:53:58.297237 (+ 1286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:58.298017 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:58.299301 (+ 1284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:58.299711 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:58.299819 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:58.303848 (+ 4029us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:53:58.303862 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:53:58.303876 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:53:58.303909 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:53:58.306484 (+ 2575us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:53:58.307143 (+ 659us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:53:58.307151 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:53:58.307153 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:53:58.307276 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:53:58.307628 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:53:58.307633 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:53:58.307635 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:53:58.308017 (+ 382us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:53:58.308307 (+ 290us) server_negotiation.cc:1035] Waiting for connection context
0809 22:53:58.308617 (+ 310us) server_negotiation.cc:299] Negotiation successful
0809 22:53:58.308827 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260809 22:53:58.309839 32756 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:53:58.397200 15814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskItYPJR/build/asan/bin/kudu with pid 32584
I20260809 22:53:58.418337 15814 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskItYPJR/build/asan/bin/kudu pid 305
Aug 09 22:53:58 dist-test-slave-ncp9 krb5kdc[32227](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316034, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.113.131@KRBTEST.COM
I20260809 22:53:58.432099 515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:53:58.419065 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42554 (local address 127.15.113.131:33765)
0809 22:53:58.419257 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0809 22:53:58.419262 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:53:58.419281 (+ 19us) server_negotiation.cc:407] Connection header received
0809 22:53:58.419386 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:53:58.419390 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:53:58.419454 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:53:58.419525 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:53:58.420690 (+ 1165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:58.421463 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:58.422645 (+ 1182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:53:58.423006 (+ 361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:53:58.423091 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:53:58.427104 (+ 4013us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:53:58.427120 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:53:58.427137 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:53:58.427168 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:53:58.429855 (+ 2687us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:53:58.430433 (+ 578us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:53:58.430440 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:53:58.430442 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:53:58.430564 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:53:58.430846 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:53:58.430848 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:53:58.430850 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:53:58.431185 (+ 335us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:53:58.431398 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0809 22:53:58.431610 (+ 212us) server_negotiation.cc:299] Negotiation successful
0809 22:53:58.431771 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260809 22:53:58.432715 481 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:53:58.519671 15814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskItYPJR/build/asan/bin/kudu with pid 305
I20260809 22:53:58.541363 15814 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskItYPJR/build/asan/bin/kudu pid 32243
I20260809 22:53:58.542491 32342 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:53:58.651018 15814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskItYPJR/build/asan/bin/kudu with pid 32243
2026-08-09T22:53:58Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (5716 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315935617919-15814-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 22:53:58 dist-test-slave-ncp9 krb5kdc[567](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 22:53:58 dist-test-slave-ncp9 krb5kdc[567](info): set up 2 sockets
Aug 09 22:53:58 dist-test-slave-ncp9 krb5kdc[567](info): commencing operation
krb5kdc: starting...
W20260809 22:53:59.881687 15814 mini_kdc.cc:121] Time spent starting KDC: real 1.183s user 0.006s sys 0.032s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315935617919-15814-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315935617919-15814-0/krb5kdc/test-user.keytab.
Aug 09 22:54:00 dist-test-slave-ncp9 krb5kdc[567](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316040, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T22:54:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T22:54:00Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.15.113.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.15.113.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.15.113.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315935617919-15814-0/krb5kdc/rangeradmin_127.15.113.148@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.15.113.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315935617919-15814-0/krb5kdc/rangeradmin_127.15.113.148@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.15.113.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.15.113.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.15.113.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315935617919-15814-0/krb5kdc/rangerlookup_127.15.113.148@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.15.113.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315935617919-15814-0/krb5kdc/rangerlookup_127.15.113.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.15.113.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.15.113.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.15.113.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315935617919-15814-0/krb5kdc/HTTP_127.15.113.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.15.113.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315935617919-15814-0/krb5kdc/HTTP_127.15.113.148@KRBTEST.COM.keytab.
I20260809 22:54:00.255748 15814 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315935617919-15814-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskItYPJR/build/asan/bin/postgres/pg_ctl -D /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315935617919-15814-0/minicluster-data/postgres -l logfile start
2026-08-09 22:54:02.606 UTC [611] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-09 22:54:02.606 UTC [611] LOG: listening on IPv4 address "127.15.113.148", port 52073
2026-08-09 22:54:02.609 UTC [611] LOG: listening on Unix socket "/tmp/.s.PGSQL.52073"
2026-08-09 22:54:02.630 UTC [614] LOG: database system was shut down at 2026-08-09 22:54:01 UTC
2026-08-09 22:54:02.634 UTC [611] LOG: database system is ready to accept connections
I20260809 22:54:03.709024 15814 mini_postgres.cc:96] Postgres bound to 52073
2026-08-09 22:54:03.733 UTC [625] FATAL: database "slave" does not exist
127.15.113.148:52073 - accepting connections
I20260809 22:54:03.734401 15814 mini_ranger.cc:161] Starting Ranger...
I20260809 22:54:03.762267 15814 mini_ranger.cc:84] Created miniranger Postgres user
I20260809 22:54:04.214951 15814 mini_ranger.cc:87] Created ranger Postgres database
I20260809 22:54:04.215097 15814 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315935617919-15814-0/minicluster-data/ranger-admin
2026-08-09 22:54:04,501 [I] DB FLAVOR :POSTGRES
2026-08-09 22:54:04,502 [I] --------- Verifying Ranger DB connection ---------
2026-08-09 22:54:04,502 [I] Checking connection..
2026-08-09 22:54:04,502 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskItYPJR/build/asan/bin/postgresql.jar:/tmp/dist-test-taskItYPJR/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.113.148:52073/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-09 22:54:04,750 [I] Checking connection passed.
2026-08-09 22:54:04,751 [I] --------- Verifying version history table ---------
2026-08-09 22:54:04,751 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskItYPJR/build/asan/bin/postgresql.jar:/tmp/dist-test-taskItYPJR/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.113.148:52073/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-09 22:54:05,008 [I] Table x_db_version_h does not exist in database ranger
2026-08-09 22:54:05,008 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskItYPJR/build/asan/bin/postgresql.jar:/tmp/dist-test-taskItYPJR/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.113.148:52073/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-09 22:54:05,273 [I] Table x_db_version_h does not exist in database ranger
2026-08-09 22:54:05,273 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-09 22:54:05,273 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskItYPJR/build/asan/bin/postgresql.jar:/tmp/dist-test-taskItYPJR/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.113.148:52073/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskItYPJR/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-09 22:54:05.516 UTC [724] WARNING: there is no transaction in progress
2026-08-09 22:54:05,522 [I] create_dbversion_catalog.sql file imported successfully
2026-08-09 22:54:05,523 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskItYPJR/build/asan/bin/postgresql.jar:/tmp/dist-test-taskItYPJR/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.113.148:52073/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-09 22:54:05,781 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-09 22:54:05,781 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-09 22:54:05,781 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskItYPJR/build/asan/bin/postgresql.jar:/tmp/dist-test-taskItYPJR/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.113.148:52073/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-09 22:54:06,036 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskItYPJR/build/asan/bin/postgresql.jar:/tmp/dist-test-taskItYPJR/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.113.148:52073/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-09 22:54:06,284 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskItYPJR/build/asan/bin/postgresql.jar:/tmp/dist-test-taskItYPJR/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.113.148:52073/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-ncp9.c.gcp-upstream.internal','N') ;"
2026-08-09 22:54:06,547 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskItYPJR/build/asan/bin/postgresql.jar:/tmp/dist-test-taskItYPJR/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.113.148:52073/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-09 22:54:06,812 [I] Table x_portal_user does not exist in database ranger
2026-08-09 22:54:06,812 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskItYPJR/build/asan/bin/postgresql.jar:/tmp/dist-test-taskItYPJR/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.113.148:52073/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-09 22:54:07,059 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-09 22:54:07,059 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskItYPJR/build/asan/bin/postgresql.jar:/tmp/dist-test-taskItYPJR/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.113.148:52073/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-09 22:54:07,292 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskItYPJR/build/asan/bin/postgresql.jar:/tmp/dist-test-taskItYPJR/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.113.148:52073/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-09 22:54:07,526 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-09 22:54:07,527 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskItYPJR/build/asan/bin/postgresql.jar:/tmp/dist-test-taskItYPJR/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.113.148:52073/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskItYPJR/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-09 22:54:08.399 UTC [901] WARNING: there is no transaction in progress
2026-08-09 22:54:08.413 UTC [901] WARNING: there is no transaction in progress
2026-08-09 22:54:08.427 UTC [901] WARNING: there is no transaction in progress
2026-08-09 22:54:08.440 UTC [901] WARNING: there is no transaction in progress
2026-08-09 22:54:08.456 UTC [901] WARNING: there is no transaction in progress
2026-08-09 22:54:08.471 UTC [901] WARNING: there is no transaction in progress
2026-08-09 22:54:08.521 UTC [901] WARNING: there is no transaction in progress
2026-08-09 22:54:08.527 UTC [901] WARNING: there is no transaction in progress
2026-08-09 22:54:08.540 UTC [901] WARNING: there is no transaction in progress
2026-08-09 22:54:08.549 UTC [901] WARNING: there is no transaction in progress
2026-08-09 22:54:08.558 UTC [901] WARNING: there is no transaction in progress
2026-08-09 22:54:08.570 UTC [901] WARNING: there is no transaction in progress
2026-08-09 22:54:08.578 UTC [901] WARNING: there is no transaction in progress
2026-08-09 22:54:08.584 UTC [901] WARNING: there is no transaction in progress
2026-08-09 22:54:08.590 UTC [901] WARNING: there is no transaction in progress
2026-08-09 22:54:08.596 UTC [901] WARNING: there is no transaction in progress
2026-08-09 22:54:09,128 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-09 22:54:09,128 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskItYPJR/build/asan/bin/postgresql.jar:/tmp/dist-test-taskItYPJR/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.113.148:52073/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-09 22:54:09,368 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-09 22:54:09,368 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskItYPJR/build/asan/bin/postgresql.jar:/tmp/dist-test-taskItYPJR/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.113.148:52073/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-09 22:54:09,624 [I] Table x_portal_user already exists in database 'ranger'
2026-08-09 22:54:09,625 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskItYPJR/build/asan/bin/postgresql.jar:/tmp/dist-test-taskItYPJR/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.113.148:52073/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-09 22:54:09,880 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-09 22:54:09,881 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskItYPJR/build/asan/bin/postgresql.jar:/tmp/dist-test-taskItYPJR/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.113.148:52073/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-09 22:54:10,135 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskItYPJR/build/asan/bin/postgresql.jar:/tmp/dist-test-taskItYPJR/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.113.148:52073/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-09 22:54:10,365 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskItYPJR/build/asan/bin/postgresql.jar:/tmp/dist-test-taskItYPJR/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.113.148:52073/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-ncp9.c.gcp-upstream.internal';"
2026-08-09 22:54:10,612 [I] CORE_DB_SCHEMA import status has been updated
2026-08-09 22:54:10,613 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskItYPJR/build/asan/bin/postgresql.jar:/tmp/dist-test-taskItYPJR/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.113.148:52073/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-09 22:54:10,866 [I] DB_PATCHES have already been applied
I20260809 22:54:10.872565 15814 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315935617919-15814-0/minicluster-data/ranger-admin:/tmp/dist-test-taskItYPJR/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskItYPJR/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskItYPJR/build/asan/bin/postgresql.jar:/tmp/dist-test-taskItYPJR/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260809 22:54:10.872692 15814 mini_ranger.cc:193] Using host: 127.15.113.148
I20260809 22:54:10.895397 15814 mini_ranger.cc:227] Ranger admin URL: http://127.15.113.148:45477
Aug 09, 2026 10:54:11 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 09, 2026 10:54:11 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 09, 2026 10:54:11 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315935617919-15814-0/minicluster-data/ranger-admin/ews/webapp
Aug 09, 2026 10:54:11 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315935617919-15814-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 09, 2026 10:54:11 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315935617919-15814-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 09, 2026 10:54:11 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315935617919-15814-0/minicluster-data/ranger-admin/ews/webapp].
Aug 09, 2026 10:54:11 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260809 22:54:12.157506 15814 mini_ranger.cc:160] Time spent starting Ranger: real 8.423s user 0.000s sys 0.061s
Aug 09, 2026 10:54:31 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 09, 2026 10:54:31 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 09, 2026 10:54:31 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 09, 2026 10:54:31 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 09, 2026 10:54:31 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 09, 2026 10:54:31 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 09, 2026 10:54:31 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 09, 2026 10:54:31 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 09, 2026 10:54:31 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 09, 2026 10:54:31 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 09, 2026 10:54:31 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 09, 2026 10:54:31 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 09, 2026 10:54:31 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 09, 2026 10:54:31 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 09, 2026 10:54:31 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 09, 2026 10:54:31 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 09, 2026 10:54:31 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 09, 2026 10:54:32 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260809 22:54:32.807786 15814 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.15.113.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.15.113.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.15.113.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315935617919-15814-0/krb5kdc/rangerkms_127.15.113.148@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.15.113.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315935617919-15814-0/krb5kdc/rangerkms_127.15.113.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.15.113.148@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.15.113.148@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.15.113.148@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315935617919-15814-0/krb5kdc/HTTP_127.15.113.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.15.113.148@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315935617919-15814-0/krb5kdc/HTTP_127.15.113.148@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 09 22:54:32 dist-test-slave-ncp9 krb5kdc[567](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316072, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260809 22:54:32.988729 15814 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260809 22:54:33.010936 15814 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260809 22:54:33.725988 15814 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260809 22:54:33.726132 15814 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315935617919-15814-0/minicluster-data/ranger-kms
I20260809 22:54:33.726279 15814 mini_ranger_kms.cc:227] Using postgres at 127.15.113.148:52073
2026-08-09 22:54:34,204 [I] DB FLAVOR :POSTGRES
2026-08-09 22:54:34,204 [I] --------- Verifying Ranger DB connection ---------
2026-08-09 22:54:34,204 [I] Checking connection
2026-08-09 22:54:34,204 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskItYPJR/build/asan/bin/postgresql.jar:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315935617919-15814-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.113.148:52073/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-09 22:54:34,428 [I] connection success
2026-08-09 22:54:34,428 [I] --------- Verifying Ranger DB tables ---------
2026-08-09 22:54:34,428 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-09 22:54:34,428 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskItYPJR/build/asan/bin/postgresql.jar:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315935617919-15814-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.113.148:52073/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-09 22:54:34,676 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-09 22:54:34,676 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-09 22:54:34,677 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-09 22:54:34,677 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskItYPJR/build/asan/bin/postgresql.jar:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315935617919-15814-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.113.148:52073/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315935617919-15814-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-09 22:54:34,925 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260809 22:54:35.389993 15814 mini_ranger_kms.cc:310] Created kms service
I20260809 22:54:35.604775 15814 mini_ranger_kms.cc:326] Created kudu user
I20260809 22:54:35.648106 15814 mini_ranger_kms.cc:343] Created rangerkms user
Aug 09 22:54:35 dist-test-slave-ncp9 krb5kdc[567](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316072, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.15.113.148@KRBTEST.COM
I20260809 22:54:36.046600 15814 mini_ranger_kms.cc:384] Added ranger policy
I20260809 22:54:36.047915 15814 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315935617919-15814-0/minicluster-data/ranger-kms:/tmp/dist-test-taskItYPJR/build/asan/bin/postgresql.jar:/tmp/dist-test-taskItYPJR/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskItYPJR/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260809 22:54:36.047997 15814 mini_ranger_kms.cc:240] Using host: 127.15.113.148
I20260809 22:54:36.074995 15814 mini_ranger_kms.cc:276] Ranger KMS PID: 1184
I20260809 22:54:36.075205 15814 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.15.113.148:51067
Aug 09, 2026 10:54:37 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 09, 2026 10:54:37 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 09, 2026 10:54:37 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315935617919-15814-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 09, 2026 10:54:37 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315935617919-15814-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 09, 2026 10:54:37 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315935617919-15814-0/minicluster-data/ranger-kms/ews/webapp].
Aug 09, 2026 10:54:37 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 09 22:54:38 dist-test-slave-ncp9 krb5kdc[567](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786316078, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.15.113.148@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:54:38.678064 15814 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.689s user 0.118s sys 0.402s
I20260809 22:54:38.678460 15814 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260809 22:54:38.678727 15814 mini_ranger_kms.cc:401] 127.15.113.148:51067/kms/v1/keys
Aug 09 22:54:38 dist-test-slave-ncp9 krb5kdc[567](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316078, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.15.113.148@KRBTEST.COM for HTTP/127.15.113.148@KRBTEST.COM
Aug 09 22:54:39 dist-test-slave-ncp9 krb5kdc[567](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316078, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.15.113.148@KRBTEST.COM for HTTP/127.15.113.148@KRBTEST.COM
Aug 09 22:54:41 dist-test-slave-ncp9 krb5kdc[567](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316081, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.15.113.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.113.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.113.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.190.keytab.
Entry for principal kudu/127.15.113.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.190.keytab.
I20260809 22:54:41.645979 15814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskItYPJR/build/asan/bin/kudu
/tmp/dist-test-taskItYPJR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315935617919-15814-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315935617919-15814-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315935617919-15814-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315935617919-15814-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.113.190:43525
--webserver_interface=127.15.113.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.190.keytab
--principal=kudu/127.15.113.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.113.148:35603
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.113.190:43525
--ranger_config_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315935617919-15814-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.15.113.148:51067/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315935617919-15814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315935617919-15814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315935617919-15814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:54:42.238315 1267 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:54:42.238713 1267 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:54:42.238952 1267 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:54:42.248592 1267 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 22:54:42.248765 1267 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:54:42.248937 1267 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:54:42.249117 1267 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 22:54:42.249289 1267 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 22:54:42.262960 1267 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.113.148:35603
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315935617919-15814-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315935617919-15814-0/minicluster-data/master-0/wal
--ranger_kms_url=127.15.113.148:51067/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.15.113.190:43525
--ranger_config_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315935617919-15814-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.190.keytab
--principal=kudu/127.15.113.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.113.190:43525
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315935617919-15814-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.113.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315935617919-15814-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:54:42.264259 1267 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:54:42.265734 1267 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:54:42.286522 1274 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:54:42.286655 1276 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:54:42.288122 1278 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:54:42.289004 1267 server_base.cc:1034] running on GCE node
I20260809 22:54:42.289940 1267 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:54:42.291597 1267 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:54:42.292855 1267 hybrid_clock.cc:648] HybridClock initialized: now 1786316082292749 us; error 48 us; skew 500 ppm
I20260809 22:54:42.293416 1267 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:54:42 dist-test-slave-ncp9 krb5kdc[567](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316082, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.113.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:54:42.300592 1267 init.cc:377] Logged in from keytab as kudu/127.15.113.190@KRBTEST.COM (short username kudu)
I20260809 22:54:42.302910 1267 webserver.cc:460] Webserver started at http://127.15.113.190:46415/ using document root <none> and password file <none>
I20260809 22:54:42.303877 1267 fs_manager.cc:359] Metadata directory not provided
I20260809 22:54:42.304204 1267 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:54:42.304801 1267 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 22:54:42 dist-test-slave-ncp9 krb5kdc[567](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316082, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.113.190@KRBTEST.COM for HTTP/127.15.113.148@KRBTEST.COM
I20260809 22:54:42.361931 1267 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315935617919-15814-0/minicluster-data/master-0/data/instance:
uuid: "c3b85d087ce245f48007e6c17e6f7303"
format_stamp: "Formatted at 2026-08-09 22:54:42 on dist-test-slave-ncp9"
server_key: "b5a81ef4304038aa8bb27b035c1c2474"
server_key_iv: "9b2a6945091f0d93096396bd81d38397"
server_key_version: "kuduclusterkey@0"
I20260809 22:54:42.363456 1267 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315935617919-15814-0/minicluster-data/master-0/wal/instance:
uuid: "c3b85d087ce245f48007e6c17e6f7303"
format_stamp: "Formatted at 2026-08-09 22:54:42 on dist-test-slave-ncp9"
server_key: "b5a81ef4304038aa8bb27b035c1c2474"
server_key_iv: "9b2a6945091f0d93096396bd81d38397"
server_key_version: "kuduclusterkey@0"
I20260809 22:54:42.373097 1267 fs_manager.cc:691] Time spent creating directory manager: real 0.009s user 0.005s sys 0.005s
I20260809 22:54:42.396585 1293 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:54:42.398442 1267 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.003s sys 0.000s
I20260809 22:54:42.398653 1267 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315935617919-15814-0/minicluster-data/master-0/data,/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315935617919-15814-0/minicluster-data/master-0/wal
uuid: "c3b85d087ce245f48007e6c17e6f7303"
format_stamp: "Formatted at 2026-08-09 22:54:42 on dist-test-slave-ncp9"
server_key: "b5a81ef4304038aa8bb27b035c1c2474"
server_key_iv: "9b2a6945091f0d93096396bd81d38397"
server_key_version: "kuduclusterkey@0"
I20260809 22:54:42.398903 1267 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315935617919-15814-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315935617919-15814-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315935617919-15814-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:54:42.425411 1267 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:54:42.432133 1267 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:54:42.432569 1267 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:54:42.479833 1366 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.113.190:43525 every 8 connection(s)
I20260809 22:54:42.479959 1267 rpc_server.cc:307] RPC server started. Bound to: 127.15.113.190:43525
I20260809 22:54:42.482339 1267 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315935617919-15814-0/minicluster-data/master-0/data/info.pb
I20260809 22:54:42.486693 15814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskItYPJR/build/asan/bin/kudu as pid 1267
I20260809 22:54:42.486909 15814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315935617919-15814-0/minicluster-data/master-0/wal/instance
I20260809 22:54:42.487900 1367 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:54:42.487975 15814 external_mini_cluster.cc:1428] Setting key 9f8234de1a6a1280a198512976360e5e
I20260809 22:54:42.510998 1367 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c3b85d087ce245f48007e6c17e6f7303: Bootstrap starting.
Aug 09 22:54:42 dist-test-slave-ncp9 krb5kdc[567](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316081, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.113.190@KRBTEST.COM
I20260809 22:54:42.516369 1367 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c3b85d087ce245f48007e6c17e6f7303: Neither blocks nor log segments found. Creating new log.
I20260809 22:54:42.518889 1367 log.cc:824] T 00000000000000000000000000000000 P c3b85d087ce245f48007e6c17e6f7303: Log is configured to *not* fsync() on all Append() calls
I20260809 22:54:42.524523 1367 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c3b85d087ce245f48007e6c17e6f7303: No bootstrap required, opened a new log
I20260809 22:54:42.524884 1370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:54:42.490617 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55322 (local address 127.15.113.190:43525)
0809 22:54:42.492293 (+ 1676us) server_negotiation.cc:206] Beginning negotiation
0809 22:54:42.492310 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0809 22:54:42.493814 (+ 1504us) server_negotiation.cc:407] Connection header received
0809 22:54:42.497692 (+ 3878us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:54:42.497725 (+ 33us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:54:42.498377 (+ 652us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:54:42.499009 (+ 632us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:54:42.500492 (+ 1483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:54:42.503499 (+ 3007us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:54:42.505403 (+ 1904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:54:42.506895 (+ 1492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:54:42.507083 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:54:42.512669 (+ 5586us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:54:42.512685 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:54:42.512705 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:54:42.512764 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:54:42.518945 (+ 6181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:54:42.519843 (+ 898us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:54:42.519852 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:54:42.519860 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:54:42.520252 (+ 392us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:54:42.520758 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:54:42.520766 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:54:42.520770 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:54:42.521720 (+ 950us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:54:42.522261 (+ 541us) server_negotiation.cc:1035] Waiting for connection context
0809 22:54:42.522658 (+ 397us) server_negotiation.cc:299] Negotiation successful
0809 22:54:42.523119 (+ 461us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1349,"thread_start_us":969,"threads_started":1}
I20260809 22:54:42.537932 1367 raft_consensus.cc:348] T 00000000000000000000000000000000 P c3b85d087ce245f48007e6c17e6f7303 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c3b85d087ce245f48007e6c17e6f7303" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 43525 } }
I20260809 22:54:42.538293 1367 raft_consensus.cc:374] T 00000000000000000000000000000000 P c3b85d087ce245f48007e6c17e6f7303 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:54:42.538419 1367 raft_consensus.cc:729] T 00000000000000000000000000000000 P c3b85d087ce245f48007e6c17e6f7303 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c3b85d087ce245f48007e6c17e6f7303, State: Initialized, Role: FOLLOWER
I20260809 22:54:42.539142 1367 consensus_queue.cc:260] T 00000000000000000000000000000000 P c3b85d087ce245f48007e6c17e6f7303 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c3b85d087ce245f48007e6c17e6f7303" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 43525 } }
I20260809 22:54:42.539548 1367 raft_consensus.cc:388] T 00000000000000000000000000000000 P c3b85d087ce245f48007e6c17e6f7303 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 22:54:42.539785 1367 raft_consensus.cc:482] T 00000000000000000000000000000000 P c3b85d087ce245f48007e6c17e6f7303 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 22:54:42.540056 1367 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c3b85d087ce245f48007e6c17e6f7303 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:54:42.543365 1367 raft_consensus.cc:504] T 00000000000000000000000000000000 P c3b85d087ce245f48007e6c17e6f7303 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c3b85d087ce245f48007e6c17e6f7303" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 43525 } }
I20260809 22:54:42.543926 1367 leader_election.cc:302] T 00000000000000000000000000000000 P c3b85d087ce245f48007e6c17e6f7303 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c3b85d087ce245f48007e6c17e6f7303; no voters:
I20260809 22:54:42.545213 1367 leader_election.cc:288] T 00000000000000000000000000000000 P c3b85d087ce245f48007e6c17e6f7303 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 22:54:42.545401 1373 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c3b85d087ce245f48007e6c17e6f7303 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 22:54:42.546337 1373 raft_consensus.cc:686] T 00000000000000000000000000000000 P c3b85d087ce245f48007e6c17e6f7303 [term 1 LEADER]: Becoming Leader. State: Replica: c3b85d087ce245f48007e6c17e6f7303, State: Running, Role: LEADER
I20260809 22:54:42.547078 1373 consensus_queue.cc:237] T 00000000000000000000000000000000 P c3b85d087ce245f48007e6c17e6f7303 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c3b85d087ce245f48007e6c17e6f7303" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 43525 } }
I20260809 22:54:42.549988 1373 sys_catalog.cc:455] T 00000000000000000000000000000000 P c3b85d087ce245f48007e6c17e6f7303 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c3b85d087ce245f48007e6c17e6f7303. Latest consensus state: current_term: 1 leader_uuid: "c3b85d087ce245f48007e6c17e6f7303" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c3b85d087ce245f48007e6c17e6f7303" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 43525 } } }
I20260809 22:54:42.550576 1373 sys_catalog.cc:458] T 00000000000000000000000000000000 P c3b85d087ce245f48007e6c17e6f7303 [sys.catalog]: This master's current role is: LEADER
I20260809 22:54:42.551363 1377 sys_catalog.cc:455] T 00000000000000000000000000000000 P c3b85d087ce245f48007e6c17e6f7303 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c3b85d087ce245f48007e6c17e6f7303" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c3b85d087ce245f48007e6c17e6f7303" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 43525 } } }
I20260809 22:54:42.553110 1377 sys_catalog.cc:458] T 00000000000000000000000000000000 P c3b85d087ce245f48007e6c17e6f7303 [sys.catalog]: This master's current role is: LEADER
I20260809 22:54:42.558568 1367 sys_catalog.cc:564] T 00000000000000000000000000000000 P c3b85d087ce245f48007e6c17e6f7303 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 22:54:42.567836 1379 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 22:54:42.577976 1379 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 22:54:42.606114 1379 catalog_manager.cc:1347] Generated new cluster ID: 7598b6c852bd41769e72466e6767df5b
I20260809 22:54:42.606242 1379 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 22:54:42.638113 1379 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 22:54:42.641983 1379 catalog_manager.cc:1504] Loading token signing keys...
I20260809 22:54:42.656111 1379 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c3b85d087ce245f48007e6c17e6f7303: Generated new TSK 0
I20260809 22:54:42.657217 1379 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 22:54:42.662568 1367 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315935617919-15814-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
WARNING: no policy specified for kudu/127.15.113.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.113.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.113.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.129.keytab.
Entry for principal kudu/127.15.113.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.129.keytab.
I20260809 22:54:43.540096 15814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskItYPJR/build/asan/bin/kudu
/tmp/dist-test-taskItYPJR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315935617919-15814-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315935617919-15814-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315935617919-15814-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315935617919-15814-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.15.113.148:51067/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.113.129:0
--local_ip_for_outbound_sockets=127.15.113.129
--webserver_interface=127.15.113.129
--webserver_port=0
--tserver_master_addrs=127.15.113.190:43525
--keytab_file=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.129.keytab
--principal=kudu/127.15.113.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.113.148:35603
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315935617919-15814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315935617919-15814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315935617919-15814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 22:54:43 dist-test-slave-ncp9 krb5kdc[567](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786316083, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.113.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260809 22:54:44.117609 1433 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:54:44.118091 1433 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:54:44.118371 1433 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:54:44.131539 1433 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:54:44.131692 1433 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:54:44.131935 1433 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.113.129
I20260809 22:54:44.149947 1433 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.113.148:35603
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315935617919-15814-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315935617919-15814-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.15.113.148:51067/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.129.keytab
--principal=kudu/127.15.113.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.113.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315935617919-15814-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.113.129
--webserver_port=0
--tserver_master_addrs=127.15.113.190:43525
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.113.129
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315935617919-15814-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:54:44.152061 1433 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:54:44.154116 1433 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:54:44.176427 1448 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:54:44.180075 1447 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:54:44.184877 1433 server_base.cc:1034] running on GCE node
W20260809 22:54:44.185330 1450 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:54:44.186146 1433 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:54:44.187283 1433 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:54:44.188527 1433 hybrid_clock.cc:648] HybridClock initialized: now 1786316084188470 us; error 37 us; skew 500 ppm
I20260809 22:54:44.189030 1433 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:54:44 dist-test-slave-ncp9 krb5kdc[567](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316084, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.113.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:54:44.194379 1433 init.cc:377] Logged in from keytab as kudu/127.15.113.129@KRBTEST.COM (short username kudu)
I20260809 22:54:44.196874 1433 webserver.cc:460] Webserver started at http://127.15.113.129:39137/ using document root <none> and password file <none>
I20260809 22:54:44.197882 1433 fs_manager.cc:359] Metadata directory not provided
I20260809 22:54:44.198117 1433 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:54:44.198551 1433 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 22:54:44 dist-test-slave-ncp9 krb5kdc[567](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316084, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.113.129@KRBTEST.COM for HTTP/127.15.113.148@KRBTEST.COM
I20260809 22:54:44.219712 1433 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315935617919-15814-0/minicluster-data/ts-0/data/instance:
uuid: "8367c63aeaa441ba809773804a864e9f"
format_stamp: "Formatted at 2026-08-09 22:54:44 on dist-test-slave-ncp9"
server_key: "d8e2faabd4a43db8d2c5c8de2cc7cfa6"
server_key_iv: "75df3867d370e69a6083b56a9776fb6d"
server_key_version: "kuduclusterkey@0"
I20260809 22:54:44.221197 1433 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315935617919-15814-0/minicluster-data/ts-0/wal/instance:
uuid: "8367c63aeaa441ba809773804a864e9f"
format_stamp: "Formatted at 2026-08-09 22:54:44 on dist-test-slave-ncp9"
server_key: "d8e2faabd4a43db8d2c5c8de2cc7cfa6"
server_key_iv: "75df3867d370e69a6083b56a9776fb6d"
server_key_version: "kuduclusterkey@0"
I20260809 22:54:44.230587 1433 fs_manager.cc:691] Time spent creating directory manager: real 0.009s user 0.002s sys 0.006s
I20260809 22:54:44.269407 1459 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:54:44.271910 1433 fs_manager.cc:725] Time spent opening block manager: real 0.023s user 0.003s sys 0.007s
I20260809 22:54:44.272171 1433 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315935617919-15814-0/minicluster-data/ts-0/data,/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315935617919-15814-0/minicluster-data/ts-0/wal
uuid: "8367c63aeaa441ba809773804a864e9f"
format_stamp: "Formatted at 2026-08-09 22:54:44 on dist-test-slave-ncp9"
server_key: "d8e2faabd4a43db8d2c5c8de2cc7cfa6"
server_key_iv: "75df3867d370e69a6083b56a9776fb6d"
server_key_version: "kuduclusterkey@0"
I20260809 22:54:44.272516 1433 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315935617919-15814-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315935617919-15814-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315935617919-15814-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:54:44.310401 1433 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:54:44.318059 1433 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:54:44.318742 1433 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:54:44.322691 1433 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:54:44.326215 1433 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:54:44.326385 1433 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:54:44.326678 1433 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:54:44.326836 1433 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 22:54:44 dist-test-slave-ncp9 krb5kdc[567](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316083, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.113.190@KRBTEST.COM for HTTP/127.15.113.148@KRBTEST.COM
I20260809 22:54:44.594406 1433 rpc_server.cc:307] RPC server started. Bound to: 127.15.113.129:44415
I20260809 22:54:44.594487 1632 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.113.129:44415 every 8 connection(s)
I20260809 22:54:44.596632 1433 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315935617919-15814-0/minicluster-data/ts-0/data/info.pb
I20260809 22:54:44.601409 15814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskItYPJR/build/asan/bin/kudu as pid 1433
I20260809 22:54:44.601626 15814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315935617919-15814-0/minicluster-data/ts-0/wal/instance
I20260809 22:54:44.602677 15814 external_mini_cluster.cc:1428] Setting key f2c8d081fe8e1792f8efe2f406ede58c
Aug 09 22:54:44 dist-test-slave-ncp9 krb5kdc[567](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316084, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.113.129@KRBTEST.COM for kudu/127.15.113.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.15.113.130@KRBTEST.COM; defaulting to no policy
I20260809 22:54:44.667452 1637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:54:44.600495 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.113.129:35299 (local address 127.15.113.190:43525)
0809 22:54:44.624998 (+ 24503us) server_negotiation.cc:206] Beginning negotiation
0809 22:54:44.625005 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:54:44.632943 (+ 7938us) server_negotiation.cc:407] Connection header received
0809 22:54:44.634596 (+ 1653us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:54:44.634602 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:54:44.634696 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:54:44.634771 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:54:44.638311 (+ 3540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:54:44.639122 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:54:44.642538 (+ 3416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:54:44.643315 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:54:44.643412 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:54:44.658268 (+ 14856us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:54:44.658279 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:54:44.658284 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:54:44.658337 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:54:44.663751 (+ 5414us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:54:44.664587 (+ 836us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:54:44.664593 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:54:44.664597 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:54:44.664806 (+ 209us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:54:44.665328 (+ 522us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:54:44.665333 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:54:44.665336 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:54:44.665742 (+ 406us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:54:44.665878 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0809 22:54:44.666702 (+ 824us) server_negotiation.cc:299] Negotiation successful
0809 22:54:44.666927 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22923,"thread_start_us":22733,"threads_started":1}
I20260809 22:54:44.668591 1638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:54:44.601285 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:43525 (local address 127.15.113.129:35299)
0809 22:54:44.630531 (+ 29246us) negotiation.cc:107] Waiting for socket to connect
0809 22:54:44.630606 (+ 75us) client_negotiation.cc:174] Beginning negotiation
0809 22:54:44.634226 (+ 3620us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:54:44.635427 (+ 1201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:54:44.635544 (+ 117us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:54:44.636680 (+ 1136us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:54:44.638088 (+ 1408us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:54:44.638119 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:54:44.639327 (+ 1208us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:54:44.639332 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:54:44.642303 (+ 2971us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:54:44.642356 (+ 53us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:54:44.643871 (+ 1515us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:54:44.643880 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:54:44.644333 (+ 453us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:54:44.653304 (+ 8971us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:54:44.653375 (+ 71us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:54:44.657799 (+ 4424us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:54:44.663912 (+ 6113us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:54:44.663921 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:54:44.663937 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:54:44.664444 (+ 507us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:54:44.664942 (+ 498us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:54:44.664945 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:54:44.664948 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:54:44.665193 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:54:44.665874 (+ 681us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:54:44.665883 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:54:44.666319 (+ 436us) client_negotiation.cc:769] Sending connection context
0809 22:54:44.666612 (+ 293us) client_negotiation.cc:240] Negotiation successful
0809 22:54:44.667234 (+ 622us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":28773,"thread_start_us":28492,"threads_started":1}
I20260809 22:54:44.670318 1634 heartbeater.cc:347] Connected to a master server at 127.15.113.190:43525
I20260809 22:54:44.670673 1634 heartbeater.cc:464] Registering TS with master...
I20260809 22:54:44.672123 1634 heartbeater.cc:511] Master 127.15.113.190:43525 requested a full tablet report, sending...
I20260809 22:54:44.674487 1316 ts_manager.cc:194] Registered new tserver with Master: 8367c63aeaa441ba809773804a864e9f (127.15.113.129:44415)
I20260809 22:54:44.676261 1316 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.113.129@KRBTEST.COM'} at 127.15.113.129:35299
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.113.130@KRBTEST.COM" created.
Aug 09 22:54:44 dist-test-slave-ncp9 krb5kdc[567](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316083, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.113.190@KRBTEST.COM for HTTP/127.15.113.148@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.113.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.130.keytab.
Entry for principal kudu/127.15.113.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.130.keytab.
W20260809 22:54:45.098022 15814 mini_kdc.cc:262] Time spent creating service keytab for kudu/127.15.113.130: real 0.472s user 0.003s sys 0.023s
I20260809 22:54:45.099078 15814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskItYPJR/build/asan/bin/kudu
/tmp/dist-test-taskItYPJR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315935617919-15814-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315935617919-15814-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315935617919-15814-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315935617919-15814-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.15.113.148:51067/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.113.130:0
--local_ip_for_outbound_sockets=127.15.113.130
--webserver_interface=127.15.113.130
--webserver_port=0
--tserver_master_addrs=127.15.113.190:43525
--keytab_file=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.130.keytab
--principal=kudu/127.15.113.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.113.148:35603
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315935617919-15814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315935617919-15814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315935617919-15814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 22:54:45.680070 1634 heartbeater.cc:503] Master 127.15.113.190:43525 was elected leader, sending a full tablet report...
W20260809 22:54:45.690920 1643 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:54:45.691238 1643 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:54:45.691399 1643 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:54:45.700186 1643 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:54:45.700299 1643 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:54:45.700479 1643 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.113.130
I20260809 22:54:45.712232 1643 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.113.148:35603
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315935617919-15814-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315935617919-15814-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.15.113.148:51067/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.130.keytab
--principal=kudu/127.15.113.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.113.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315935617919-15814-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.113.130
--webserver_port=0
--tserver_master_addrs=127.15.113.190:43525
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.113.130
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315935617919-15814-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:54:45.713735 1643 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:54:45.715231 1643 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:54:45.730268 1659 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:54:45.730944 1660 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:54:45.731185 1665 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:54:45.734397 1643 server_base.cc:1034] running on GCE node
I20260809 22:54:45.735085 1643 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:54:45.736060 1643 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:54:45.737277 1643 hybrid_clock.cc:648] HybridClock initialized: now 1786316085737199 us; error 51 us; skew 500 ppm
I20260809 22:54:45.737588 1643 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:54:45 dist-test-slave-ncp9 krb5kdc[567](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316085, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.113.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:54:45.741039 1643 init.cc:377] Logged in from keytab as kudu/127.15.113.130@KRBTEST.COM (short username kudu)
I20260809 22:54:45.742913 1643 webserver.cc:460] Webserver started at http://127.15.113.130:41661/ using document root <none> and password file <none>
I20260809 22:54:45.743736 1643 fs_manager.cc:359] Metadata directory not provided
I20260809 22:54:45.743933 1643 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:54:45.744364 1643 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 22:54:45 dist-test-slave-ncp9 krb5kdc[567](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316085, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.113.130@KRBTEST.COM for HTTP/127.15.113.148@KRBTEST.COM
I20260809 22:54:45.761723 1643 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315935617919-15814-0/minicluster-data/ts-1/data/instance:
uuid: "106d3871856b48ebb6a2c8b60746ec2c"
format_stamp: "Formatted at 2026-08-09 22:54:45 on dist-test-slave-ncp9"
server_key: "2f7ded2128e9fd41e6db930565a4960d"
server_key_iv: "dba08ad9b91259bb3c838ffb1635df8f"
server_key_version: "kuduclusterkey@0"
I20260809 22:54:45.762724 1643 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315935617919-15814-0/minicluster-data/ts-1/wal/instance:
uuid: "106d3871856b48ebb6a2c8b60746ec2c"
format_stamp: "Formatted at 2026-08-09 22:54:45 on dist-test-slave-ncp9"
server_key: "2f7ded2128e9fd41e6db930565a4960d"
server_key_iv: "dba08ad9b91259bb3c838ffb1635df8f"
server_key_version: "kuduclusterkey@0"
I20260809 22:54:45.769271 1643 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.002s sys 0.006s
I20260809 22:54:45.786024 1674 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:54:45.787652 1643 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260809 22:54:45.787838 1643 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315935617919-15814-0/minicluster-data/ts-1/data,/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315935617919-15814-0/minicluster-data/ts-1/wal
uuid: "106d3871856b48ebb6a2c8b60746ec2c"
format_stamp: "Formatted at 2026-08-09 22:54:45 on dist-test-slave-ncp9"
server_key: "2f7ded2128e9fd41e6db930565a4960d"
server_key_iv: "dba08ad9b91259bb3c838ffb1635df8f"
server_key_version: "kuduclusterkey@0"
I20260809 22:54:45.788158 1643 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315935617919-15814-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315935617919-15814-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315935617919-15814-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:54:45.804266 1643 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:54:45.809331 1643 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:54:45.809975 1643 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:54:45.811575 1643 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:54:45.813722 1643 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:54:45.813863 1643 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:54:45.814110 1643 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:54:45.814273 1643 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:54:45.914036 1643 rpc_server.cc:307] RPC server started. Bound to: 127.15.113.130:41727
I20260809 22:54:45.915395 1643 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315935617919-15814-0/minicluster-data/ts-1/data/info.pb
I20260809 22:54:45.918444 1835 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.113.130:41727 every 8 connection(s)
I20260809 22:54:45.920046 15814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskItYPJR/build/asan/bin/kudu as pid 1643
I20260809 22:54:45.920213 15814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315935617919-15814-0/minicluster-data/ts-1/wal/instance
I20260809 22:54:45.920739 15814 external_mini_cluster.cc:1428] Setting key 0557c70b02c3d76bccf1b92f4f8ebc27
Aug 09 22:54:45 dist-test-slave-ncp9 krb5kdc[567](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316085, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.113.130@KRBTEST.COM for kudu/127.15.113.190@KRBTEST.COM
I20260809 22:54:45.953917 1838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:54:45.920468 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.113.130:44117 (local address 127.15.113.190:43525)
0809 22:54:45.933598 (+ 13130us) server_negotiation.cc:206] Beginning negotiation
0809 22:54:45.933604 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:54:45.933626 (+ 22us) server_negotiation.cc:407] Connection header received
0809 22:54:45.933687 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:54:45.933692 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:54:45.933768 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:54:45.933878 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:54:45.935907 (+ 2029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:54:45.936981 (+ 1074us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:54:45.939453 (+ 2472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:54:45.940307 (+ 854us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:54:45.940437 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:54:45.945653 (+ 5216us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:54:45.945659 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:54:45.945664 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:54:45.945722 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:54:45.949860 (+ 4138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:54:45.950884 (+ 1024us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:54:45.950890 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:54:45.950893 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:54:45.951074 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:54:45.951673 (+ 599us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:54:45.951677 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:54:45.951681 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:54:45.952082 (+ 401us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:54:45.952237 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0809 22:54:45.953198 (+ 961us) server_negotiation.cc:299] Negotiation successful
0809 22:54:45.953419 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12973,"thread_start_us":12736,"threads_started":1}
I20260809 22:54:45.955127 1839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:54:45.921255 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:43525 (local address 127.15.113.130:44117)
0809 22:54:45.922633 (+ 1378us) negotiation.cc:107] Waiting for socket to connect
0809 22:54:45.922683 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0809 22:54:45.929083 (+ 6400us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:54:45.934183 (+ 5100us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:54:45.934193 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:54:45.934934 (+ 741us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:54:45.935727 (+ 793us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:54:45.935749 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:54:45.937127 (+ 1378us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:54:45.937131 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:54:45.939235 (+ 2104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:54:45.939290 (+ 55us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:54:45.940664 (+ 1374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:54:45.940673 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:54:45.941090 (+ 417us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:54:45.942273 (+ 1183us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:54:45.942302 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:54:45.945321 (+ 3019us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:54:45.950017 (+ 4696us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:54:45.950026 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:54:45.950044 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:54:45.950768 (+ 724us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:54:45.951275 (+ 507us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:54:45.951280 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:54:45.951284 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:54:45.951564 (+ 280us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:54:45.952241 (+ 677us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:54:45.952248 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:54:45.952708 (+ 460us) client_negotiation.cc:769] Sending connection context
0809 22:54:45.953083 (+ 375us) client_negotiation.cc:240] Negotiation successful
0809 22:54:45.953653 (+ 570us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":942,"thread_start_us":776,"threads_started":1}
WARNING: no policy specified for kudu/127.15.113.131@KRBTEST.COM; defaulting to no policy
I20260809 22:54:45.956936 1836 heartbeater.cc:347] Connected to a master server at 127.15.113.190:43525
I20260809 22:54:45.957252 1836 heartbeater.cc:464] Registering TS with master...
I20260809 22:54:45.958191 1836 heartbeater.cc:511] Master 127.15.113.190:43525 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.113.131@KRBTEST.COM" created.
I20260809 22:54:45.959894 1316 ts_manager.cc:194] Registered new tserver with Master: 106d3871856b48ebb6a2c8b60746ec2c (127.15.113.130:41727)
I20260809 22:54:45.960995 1316 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.113.130@KRBTEST.COM'} at 127.15.113.130:44117
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.113.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.131.keytab.
Entry for principal kudu/127.15.113.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.131.keytab.
I20260809 22:54:45.991679 15814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskItYPJR/build/asan/bin/kudu
/tmp/dist-test-taskItYPJR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315935617919-15814-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315935617919-15814-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315935617919-15814-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315935617919-15814-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.15.113.148:51067/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.113.131:0
--local_ip_for_outbound_sockets=127.15.113.131
--webserver_interface=127.15.113.131
--webserver_port=0
--tserver_master_addrs=127.15.113.190:43525
--keytab_file=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.131.keytab
--principal=kudu/127.15.113.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.113.148:35603
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315935617919-15814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315935617919-15814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315935617919-15814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:54:46.520396 1843 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:54:46.520787 1843 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:54:46.521050 1843 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:54:46.530242 1843 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:54:46.530380 1843 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:54:46.530567 1843 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.113.131
I20260809 22:54:46.543313 1843 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.113.148:35603
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315935617919-15814-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315935617919-15814-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.15.113.148:51067/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.131.keytab
--principal=kudu/127.15.113.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.113.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315935617919-15814-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.113.131
--webserver_port=0
--tserver_master_addrs=127.15.113.190:43525
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.113.131
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315935617919-15814-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:54:46.544859 1843 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:54:46.546314 1843 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:54:46.561911 1851 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:54:46.562650 1855 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:54:46.562855 1853 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:54:46.563659 1843 server_base.cc:1034] running on GCE node
I20260809 22:54:46.564275 1843 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:54:46.565307 1843 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:54:46.566557 1843 hybrid_clock.cc:648] HybridClock initialized: now 1786316086566454 us; error 35 us; skew 500 ppm
I20260809 22:54:46.566934 1843 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:54:46 dist-test-slave-ncp9 krb5kdc[567](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316086, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.113.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:54:46.570979 1843 init.cc:377] Logged in from keytab as kudu/127.15.113.131@KRBTEST.COM (short username kudu)
I20260809 22:54:46.572772 1843 webserver.cc:460] Webserver started at http://127.15.113.131:33071/ using document root <none> and password file <none>
I20260809 22:54:46.573565 1843 fs_manager.cc:359] Metadata directory not provided
I20260809 22:54:46.573762 1843 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:54:46.574110 1843 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 22:54:46 dist-test-slave-ncp9 krb5kdc[567](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316086, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.113.131@KRBTEST.COM for HTTP/127.15.113.148@KRBTEST.COM
I20260809 22:54:46.592507 1843 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315935617919-15814-0/minicluster-data/ts-2/data/instance:
uuid: "ceabead877694c22bd406c8f091718cf"
format_stamp: "Formatted at 2026-08-09 22:54:46 on dist-test-slave-ncp9"
server_key: "b947f0af8360852071a9ba4a750de203"
server_key_iv: "c28f699ebecaa49919258a582b7d68a0"
server_key_version: "kuduclusterkey@0"
I20260809 22:54:46.593747 1843 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315935617919-15814-0/minicluster-data/ts-2/wal/instance:
uuid: "ceabead877694c22bd406c8f091718cf"
format_stamp: "Formatted at 2026-08-09 22:54:46 on dist-test-slave-ncp9"
server_key: "b947f0af8360852071a9ba4a750de203"
server_key_iv: "c28f699ebecaa49919258a582b7d68a0"
server_key_version: "kuduclusterkey@0"
I20260809 22:54:46.602293 1843 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.006s sys 0.002s
I20260809 22:54:46.618989 1867 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:54:46.620635 1843 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.003s sys 0.001s
I20260809 22:54:46.620793 1843 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315935617919-15814-0/minicluster-data/ts-2/data,/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315935617919-15814-0/minicluster-data/ts-2/wal
uuid: "ceabead877694c22bd406c8f091718cf"
format_stamp: "Formatted at 2026-08-09 22:54:46 on dist-test-slave-ncp9"
server_key: "b947f0af8360852071a9ba4a750de203"
server_key_iv: "c28f699ebecaa49919258a582b7d68a0"
server_key_version: "kuduclusterkey@0"
I20260809 22:54:46.621030 1843 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315935617919-15814-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315935617919-15814-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315935617919-15814-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:54:46.645635 1843 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:54:46.650712 1843 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:54:46.651304 1843 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:54:46.652861 1843 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:54:46.654596 1843 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:54:46.654743 1843 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:54:46.654902 1843 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:54:46.654978 1843 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:54:46.734265 1843 rpc_server.cc:307] RPC server started. Bound to: 127.15.113.131:45815
I20260809 22:54:46.734587 2035 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.113.131:45815 every 8 connection(s)
I20260809 22:54:46.736090 1843 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315935617919-15814-0/minicluster-data/ts-2/data/info.pb
I20260809 22:54:46.743765 15814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskItYPJR/build/asan/bin/kudu as pid 1843
I20260809 22:54:46.743981 15814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315935617919-15814-0/minicluster-data/ts-2/wal/instance
I20260809 22:54:46.744630 15814 external_mini_cluster.cc:1428] Setting key 936dda85a94aaf0a5b8390605f27c829
Aug 09 22:54:46 dist-test-slave-ncp9 krb5kdc[567](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316086, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.113.131@KRBTEST.COM for kudu/127.15.113.190@KRBTEST.COM
I20260809 22:54:46.762871 2038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:54:46.739295 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.113.131:36887 (local address 127.15.113.190:43525)
0809 22:54:46.740163 (+ 868us) server_negotiation.cc:206] Beginning negotiation
0809 22:54:46.740167 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 22:54:46.741233 (+ 1066us) server_negotiation.cc:407] Connection header received
0809 22:54:46.744103 (+ 2870us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:54:46.744107 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:54:46.744187 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:54:46.744302 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:54:46.746792 (+ 2490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:54:46.748247 (+ 1455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:54:46.750471 (+ 2224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:54:46.751213 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:54:46.751362 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:54:46.755316 (+ 3954us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:54:46.755321 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:54:46.755325 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:54:46.755400 (+ 75us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:54:46.759471 (+ 4071us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:54:46.760351 (+ 880us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:54:46.760357 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:54:46.760360 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:54:46.760594 (+ 234us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:54:46.761069 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:54:46.761072 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:54:46.761075 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:54:46.761462 (+ 387us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:54:46.761577 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0809 22:54:46.762162 (+ 585us) server_negotiation.cc:299] Negotiation successful
0809 22:54:46.762383 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":753,"thread_start_us":532,"threads_started":1}
I20260809 22:54:46.763407 2041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:54:46.740029 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:43525 (local address 127.15.113.131:36887)
0809 22:54:46.741064 (+ 1035us) negotiation.cc:107] Waiting for socket to connect
0809 22:54:46.741108 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 22:54:46.743764 (+ 2656us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:54:46.744643 (+ 879us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:54:46.744655 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:54:46.745565 (+ 910us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:54:46.746613 (+ 1048us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:54:46.746636 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:54:46.748384 (+ 1748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:54:46.748389 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:54:46.750276 (+ 1887us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:54:46.750300 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:54:46.751492 (+ 1192us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:54:46.751499 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:54:46.751723 (+ 224us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:54:46.752410 (+ 687us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:54:46.752467 (+ 57us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:54:46.755112 (+ 2645us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:54:46.759601 (+ 4489us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:54:46.759609 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:54:46.759624 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:54:46.760228 (+ 604us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:54:46.760711 (+ 483us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:54:46.760714 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:54:46.760717 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:54:46.760967 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:54:46.761557 (+ 590us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:54:46.761562 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:54:46.761842 (+ 280us) client_negotiation.cc:769] Sending connection context
0809 22:54:46.762046 (+ 204us) client_negotiation.cc:240] Negotiation successful
0809 22:54:46.762415 (+ 369us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":689,"thread_start_us":435,"threads_started":1}
I20260809 22:54:46.764693 2036 heartbeater.cc:347] Connected to a master server at 127.15.113.190:43525
I20260809 22:54:46.765089 2036 heartbeater.cc:464] Registering TS with master...
I20260809 22:54:46.766021 2036 heartbeater.cc:511] Master 127.15.113.190:43525 requested a full tablet report, sending...
I20260809 22:54:46.767541 1316 ts_manager.cc:194] Registered new tserver with Master: ceabead877694c22bd406c8f091718cf (127.15.113.131:45815)
I20260809 22:54:46.768697 1316 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.113.131@KRBTEST.COM'} at 127.15.113.131:36887
I20260809 22:54:46.773047 15814 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 22:54:46.793977 2038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:54:46.778843 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55338 (local address 127.15.113.190:43525)
0809 22:54:46.779011 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0809 22:54:46.779015 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 22:54:46.779681 (+ 666us) server_negotiation.cc:407] Connection header received
0809 22:54:46.779827 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:54:46.779831 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:54:46.779898 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:54:46.779972 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:54:46.781419 (+ 1447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:54:46.782233 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:54:46.784064 (+ 1831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:54:46.784519 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:54:46.784607 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:54:46.788093 (+ 3486us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:54:46.788099 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:54:46.788102 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:54:46.788133 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:54:46.791418 (+ 3285us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:54:46.792104 (+ 686us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:54:46.792107 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:54:46.792109 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:54:46.792222 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:54:46.792697 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:54:46.792700 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:54:46.792701 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:54:46.793021 (+ 320us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:54:46.793196 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0809 22:54:46.793562 (+ 366us) server_negotiation.cc:299] Negotiation successful
0809 22:54:46.793732 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260809 22:54:46.798761 1316 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55338:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 22:54:46.809830 1316 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 22:54:46.857056 2064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:54:46.827233 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45522 (local address 127.15.113.130:41727)
0809 22:54:46.846374 (+ 19141us) server_negotiation.cc:206] Beginning negotiation
0809 22:54:46.846383 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 22:54:46.846413 (+ 30us) server_negotiation.cc:407] Connection header received
0809 22:54:46.846518 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:54:46.846526 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:54:46.846788 (+ 262us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:54:46.847016 (+ 228us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:54:46.848617 (+ 1601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:54:46.850542 (+ 1925us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:54:46.854373 (+ 3831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:54:46.855519 (+ 1146us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:54:46.855682 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:54:46.855867 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0809 22:54:46.856309 (+ 442us) server_negotiation.cc:299] Negotiation successful
0809 22:54:46.856507 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18978,"thread_start_us":19079,"threads_started":1}
I20260809 22:54:46.857332 2065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:54:46.826989 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.130:41727 (local address 127.0.0.1:45522)
0809 22:54:46.835546 (+ 8557us) negotiation.cc:107] Waiting for socket to connect
0809 22:54:46.835604 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0809 22:54:46.835868 (+ 264us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:54:46.847019 (+ 11151us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:54:46.847024 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:54:46.847050 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:54:46.848381 (+ 1331us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:54:46.848397 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:54:46.850713 (+ 2316us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:54:46.850717 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:54:46.854237 (+ 3520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:54:46.854249 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:54:46.855816 (+ 1567us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:54:46.855823 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:54:46.856060 (+ 237us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:54:46.856102 (+ 42us) client_negotiation.cc:769] Sending connection context
0809 22:54:46.856251 (+ 149us) client_negotiation.cc:240] Negotiation successful
0809 22:54:46.856434 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8138,"mutex_wait_us":20,"thread_start_us":13752,"threads_started":1}
I20260809 22:54:46.857543 2066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:54:46.827924 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45552 (local address 127.15.113.129:44415)
0809 22:54:46.843790 (+ 15866us) server_negotiation.cc:206] Beginning negotiation
0809 22:54:46.843799 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 22:54:46.843831 (+ 32us) server_negotiation.cc:407] Connection header received
0809 22:54:46.843976 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:54:46.843987 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:54:46.844267 (+ 280us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:54:46.845530 (+ 1263us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:54:46.846383 (+ 853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:54:46.849073 (+ 2690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:54:46.853186 (+ 4113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:54:46.855431 (+ 2245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:54:46.855710 (+ 279us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:54:46.855881 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0809 22:54:46.856803 (+ 922us) server_negotiation.cc:299] Negotiation successful
0809 22:54:46.857028 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15635,"thread_start_us":15336,"threads_started":1}
I20260809 22:54:46.858661 2067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:54:46.828682 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38840 (local address 127.15.113.131:45815)
0809 22:54:46.844359 (+ 15677us) server_negotiation.cc:206] Beginning negotiation
0809 22:54:46.844366 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:54:46.844400 (+ 34us) server_negotiation.cc:407] Connection header received
0809 22:54:46.844493 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:54:46.844501 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:54:46.844765 (+ 264us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:54:46.845042 (+ 277us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:54:46.847473 (+ 2431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:54:46.849702 (+ 2229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:54:46.853992 (+ 4290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:54:46.856188 (+ 2196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:54:46.856461 (+ 273us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:54:46.856612 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0809 22:54:46.857953 (+ 1341us) server_negotiation.cc:299] Negotiation successful
0809 22:54:46.858148 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15503,"thread_start_us":15331,"threads_started":1}
I20260809 22:54:46.859272 2060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:54:46.827820 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.131:45815 (local address 127.0.0.1:38840)
0809 22:54:46.841420 (+ 13600us) negotiation.cc:107] Waiting for socket to connect
0809 22:54:46.841447 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0809 22:54:46.841575 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:54:46.845038 (+ 3463us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:54:46.845044 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:54:46.845073 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:54:46.846838 (+ 1765us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:54:46.846853 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:54:46.849841 (+ 2988us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:54:46.849847 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:54:46.853755 (+ 3908us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:54:46.853770 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:54:46.857091 (+ 3321us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:54:46.857104 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:54:46.857383 (+ 279us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:54:46.857417 (+ 34us) client_negotiation.cc:769] Sending connection context
0809 22:54:46.857505 (+ 88us) client_negotiation.cc:240] Negotiation successful
0809 22:54:46.857723 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13424,"thread_start_us":6998,"threads_started":1}
I20260809 22:54:46.859475 2068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:54:46.827680 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.129:44415 (local address 127.0.0.1:45552)
0809 22:54:46.841360 (+ 13680us) negotiation.cc:107] Waiting for socket to connect
0809 22:54:46.841381 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0809 22:54:46.841541 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:54:46.844552 (+ 3011us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:54:46.844559 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:54:46.844590 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:54:46.846114 (+ 1524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:54:46.846129 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:54:46.849205 (+ 3076us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:54:46.849210 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:54:46.852881 (+ 3671us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:54:46.852957 (+ 76us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:54:46.856130 (+ 3173us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:54:46.856138 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:54:46.856302 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:54:46.856336 (+ 34us) client_negotiation.cc:769] Sending connection context
0809 22:54:46.856416 (+ 80us) client_negotiation.cc:240] Negotiation successful
0809 22:54:46.856540 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13476,"mutex_wait_us":23,"thread_start_us":12576,"threads_started":1}
I20260809 22:54:46.862525 1745 tablet_service.cc:1467] Processing CreateTablet for tablet 87dda3c64e5e43ce952537be7eaf149a (DEFAULT_TABLE table=test-table [id=fa3bab96f50d46808dd819895c2516d4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:54:46.863353 1526 tablet_service.cc:1467] Processing CreateTablet for tablet 87dda3c64e5e43ce952537be7eaf149a (DEFAULT_TABLE table=test-table [id=fa3bab96f50d46808dd819895c2516d4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:54:46.864328 1745 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 87dda3c64e5e43ce952537be7eaf149a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:54:46.863994 1936 tablet_service.cc:1467] Processing CreateTablet for tablet 87dda3c64e5e43ce952537be7eaf149a (DEFAULT_TABLE table=test-table [id=fa3bab96f50d46808dd819895c2516d4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:54:46.865140 1526 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 87dda3c64e5e43ce952537be7eaf149a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:54:46.865477 1936 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 87dda3c64e5e43ce952537be7eaf149a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:54:46.882673 2070 tablet_bootstrap.cc:492] T 87dda3c64e5e43ce952537be7eaf149a P 8367c63aeaa441ba809773804a864e9f: Bootstrap starting.
I20260809 22:54:46.886059 2072 tablet_bootstrap.cc:492] T 87dda3c64e5e43ce952537be7eaf149a P ceabead877694c22bd406c8f091718cf: Bootstrap starting.
I20260809 22:54:46.887990 2071 tablet_bootstrap.cc:492] T 87dda3c64e5e43ce952537be7eaf149a P 106d3871856b48ebb6a2c8b60746ec2c: Bootstrap starting.
I20260809 22:54:46.889700 2070 tablet_bootstrap.cc:654] T 87dda3c64e5e43ce952537be7eaf149a P 8367c63aeaa441ba809773804a864e9f: Neither blocks nor log segments found. Creating new log.
I20260809 22:54:46.892273 2070 log.cc:824] T 87dda3c64e5e43ce952537be7eaf149a P 8367c63aeaa441ba809773804a864e9f: Log is configured to *not* fsync() on all Append() calls
I20260809 22:54:46.893185 2071 tablet_bootstrap.cc:654] T 87dda3c64e5e43ce952537be7eaf149a P 106d3871856b48ebb6a2c8b60746ec2c: Neither blocks nor log segments found. Creating new log.
I20260809 22:54:46.893290 2072 tablet_bootstrap.cc:654] T 87dda3c64e5e43ce952537be7eaf149a P ceabead877694c22bd406c8f091718cf: Neither blocks nor log segments found. Creating new log.
I20260809 22:54:46.895270 2072 log.cc:824] T 87dda3c64e5e43ce952537be7eaf149a P ceabead877694c22bd406c8f091718cf: Log is configured to *not* fsync() on all Append() calls
I20260809 22:54:46.895610 2071 log.cc:824] T 87dda3c64e5e43ce952537be7eaf149a P 106d3871856b48ebb6a2c8b60746ec2c: Log is configured to *not* fsync() on all Append() calls
I20260809 22:54:46.897562 2070 tablet_bootstrap.cc:492] T 87dda3c64e5e43ce952537be7eaf149a P 8367c63aeaa441ba809773804a864e9f: No bootstrap required, opened a new log
I20260809 22:54:46.897981 2070 ts_tablet_manager.cc:1397] T 87dda3c64e5e43ce952537be7eaf149a P 8367c63aeaa441ba809773804a864e9f: Time spent bootstrapping tablet: real 0.016s user 0.003s sys 0.010s
I20260809 22:54:46.899825 2072 tablet_bootstrap.cc:492] T 87dda3c64e5e43ce952537be7eaf149a P ceabead877694c22bd406c8f091718cf: No bootstrap required, opened a new log
I20260809 22:54:46.900224 2072 ts_tablet_manager.cc:1397] T 87dda3c64e5e43ce952537be7eaf149a P ceabead877694c22bd406c8f091718cf: Time spent bootstrapping tablet: real 0.014s user 0.004s sys 0.007s
I20260809 22:54:46.900547 2071 tablet_bootstrap.cc:492] T 87dda3c64e5e43ce952537be7eaf149a P 106d3871856b48ebb6a2c8b60746ec2c: No bootstrap required, opened a new log
I20260809 22:54:46.900971 2071 ts_tablet_manager.cc:1397] T 87dda3c64e5e43ce952537be7eaf149a P 106d3871856b48ebb6a2c8b60746ec2c: Time spent bootstrapping tablet: real 0.013s user 0.007s sys 0.004s
I20260809 22:54:46.909175 2070 raft_consensus.cc:348] T 87dda3c64e5e43ce952537be7eaf149a P 8367c63aeaa441ba809773804a864e9f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "106d3871856b48ebb6a2c8b60746ec2c" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 41727 } } peers { permanent_uuid: "8367c63aeaa441ba809773804a864e9f" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 44415 } } peers { permanent_uuid: "ceabead877694c22bd406c8f091718cf" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 45815 } }
I20260809 22:54:46.909881 2070 raft_consensus.cc:374] T 87dda3c64e5e43ce952537be7eaf149a P 8367c63aeaa441ba809773804a864e9f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:54:46.910220 2070 raft_consensus.cc:729] T 87dda3c64e5e43ce952537be7eaf149a P 8367c63aeaa441ba809773804a864e9f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8367c63aeaa441ba809773804a864e9f, State: Initialized, Role: FOLLOWER
I20260809 22:54:46.911055 2070 consensus_queue.cc:260] T 87dda3c64e5e43ce952537be7eaf149a P 8367c63aeaa441ba809773804a864e9f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "106d3871856b48ebb6a2c8b60746ec2c" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 41727 } } peers { permanent_uuid: "8367c63aeaa441ba809773804a864e9f" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 44415 } } peers { permanent_uuid: "ceabead877694c22bd406c8f091718cf" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 45815 } }
I20260809 22:54:46.913695 2070 ts_tablet_manager.cc:1428] T 87dda3c64e5e43ce952537be7eaf149a P 8367c63aeaa441ba809773804a864e9f: Time spent starting tablet: real 0.016s user 0.013s sys 0.002s
I20260809 22:54:46.915871 2072 raft_consensus.cc:348] T 87dda3c64e5e43ce952537be7eaf149a P ceabead877694c22bd406c8f091718cf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "106d3871856b48ebb6a2c8b60746ec2c" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 41727 } } peers { permanent_uuid: "8367c63aeaa441ba809773804a864e9f" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 44415 } } peers { permanent_uuid: "ceabead877694c22bd406c8f091718cf" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 45815 } }
I20260809 22:54:46.916389 2072 raft_consensus.cc:374] T 87dda3c64e5e43ce952537be7eaf149a P ceabead877694c22bd406c8f091718cf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:54:46.916535 2072 raft_consensus.cc:729] T 87dda3c64e5e43ce952537be7eaf149a P ceabead877694c22bd406c8f091718cf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ceabead877694c22bd406c8f091718cf, State: Initialized, Role: FOLLOWER
I20260809 22:54:46.917279 2072 consensus_queue.cc:260] T 87dda3c64e5e43ce952537be7eaf149a P ceabead877694c22bd406c8f091718cf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "106d3871856b48ebb6a2c8b60746ec2c" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 41727 } } peers { permanent_uuid: "8367c63aeaa441ba809773804a864e9f" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 44415 } } peers { permanent_uuid: "ceabead877694c22bd406c8f091718cf" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 45815 } }
I20260809 22:54:46.918632 2071 raft_consensus.cc:348] T 87dda3c64e5e43ce952537be7eaf149a P 106d3871856b48ebb6a2c8b60746ec2c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "106d3871856b48ebb6a2c8b60746ec2c" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 41727 } } peers { permanent_uuid: "8367c63aeaa441ba809773804a864e9f" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 44415 } } peers { permanent_uuid: "ceabead877694c22bd406c8f091718cf" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 45815 } }
I20260809 22:54:46.919123 2071 raft_consensus.cc:374] T 87dda3c64e5e43ce952537be7eaf149a P 106d3871856b48ebb6a2c8b60746ec2c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:54:46.919298 2071 raft_consensus.cc:729] T 87dda3c64e5e43ce952537be7eaf149a P 106d3871856b48ebb6a2c8b60746ec2c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 106d3871856b48ebb6a2c8b60746ec2c, State: Initialized, Role: FOLLOWER
I20260809 22:54:46.920069 2071 consensus_queue.cc:260] T 87dda3c64e5e43ce952537be7eaf149a P 106d3871856b48ebb6a2c8b60746ec2c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "106d3871856b48ebb6a2c8b60746ec2c" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 41727 } } peers { permanent_uuid: "8367c63aeaa441ba809773804a864e9f" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 44415 } } peers { permanent_uuid: "ceabead877694c22bd406c8f091718cf" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 45815 } }
I20260809 22:54:46.922859 2036 heartbeater.cc:503] Master 127.15.113.190:43525 was elected leader, sending a full tablet report...
I20260809 22:54:46.923592 1836 heartbeater.cc:503] Master 127.15.113.190:43525 was elected leader, sending a full tablet report...
I20260809 22:54:46.923657 2072 ts_tablet_manager.cc:1428] T 87dda3c64e5e43ce952537be7eaf149a P ceabead877694c22bd406c8f091718cf: Time spent starting tablet: real 0.023s user 0.020s sys 0.004s
I20260809 22:54:46.924141 2071 ts_tablet_manager.cc:1428] T 87dda3c64e5e43ce952537be7eaf149a P 106d3871856b48ebb6a2c8b60746ec2c: Time spent starting tablet: real 0.023s user 0.006s sys 0.014s
I20260809 22:54:46.938421 2081 raft_consensus.cc:482] T 87dda3c64e5e43ce952537be7eaf149a P 106d3871856b48ebb6a2c8b60746ec2c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 22:54:46.938803 2081 raft_consensus.cc:504] T 87dda3c64e5e43ce952537be7eaf149a P 106d3871856b48ebb6a2c8b60746ec2c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "106d3871856b48ebb6a2c8b60746ec2c" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 41727 } } peers { permanent_uuid: "8367c63aeaa441ba809773804a864e9f" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 44415 } } peers { permanent_uuid: "ceabead877694c22bd406c8f091718cf" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 45815 } }
I20260809 22:54:46.940871 2081 leader_election.cc:288] T 87dda3c64e5e43ce952537be7eaf149a P 106d3871856b48ebb6a2c8b60746ec2c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8367c63aeaa441ba809773804a864e9f (127.15.113.129:44415), ceabead877694c22bd406c8f091718cf (127.15.113.131:45815)
I20260809 22:54:46.950745 2067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:54:46.941375 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.113.130:52075 (local address 127.15.113.131:45815)
0809 22:54:46.941598 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0809 22:54:46.941604 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:54:46.942995 (+ 1391us) server_negotiation.cc:407] Connection header received
0809 22:54:46.943210 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:54:46.943215 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:54:46.943292 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:54:46.943408 (+ 116us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:54:46.944879 (+ 1471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:54:46.946084 (+ 1205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:54:46.948451 (+ 2367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:54:46.949782 (+ 1331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:54:46.949958 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:54:46.950036 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0809 22:54:46.950274 (+ 238us) server_negotiation.cc:299] Negotiation successful
0809 22:54:46.950394 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260809 22:54:46.951045 2082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:54:46.941204 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.131:45815 (local address 127.15.113.130:52075)
0809 22:54:46.942872 (+ 1668us) negotiation.cc:107] Waiting for socket to connect
0809 22:54:46.942892 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 22:54:46.943066 (+ 174us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:54:46.943639 (+ 573us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:54:46.943643 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:54:46.943664 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:54:46.944734 (+ 1070us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:54:46.944748 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:54:46.946240 (+ 1492us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:54:46.946244 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:54:46.948256 (+ 2012us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:54:46.948269 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:54:46.949992 (+ 1723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:54:46.949999 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:54:46.950134 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:54:46.950158 (+ 24us) client_negotiation.cc:769] Sending connection context
0809 22:54:46.950239 (+ 81us) client_negotiation.cc:240] Negotiation successful
0809 22:54:46.950363 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1474,"spinlock_wait_cycles":41856,"thread_start_us":1028,"threads_started":1}
I20260809 22:54:46.952102 2066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:54:46.941168 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.113.130:55011 (local address 127.15.113.129:44415)
0809 22:54:46.941382 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0809 22:54:46.941388 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:54:46.943830 (+ 2442us) server_negotiation.cc:407] Connection header received
0809 22:54:46.944001 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:54:46.944007 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:54:46.944082 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:54:46.944183 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:54:46.945485 (+ 1302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:54:46.946649 (+ 1164us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:54:46.949723 (+ 3074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:54:46.951110 (+ 1387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:54:46.951292 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:54:46.951371 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0809 22:54:46.951704 (+ 333us) server_negotiation.cc:299] Negotiation successful
0809 22:54:46.951835 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260809 22:54:46.952193 2084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:54:46.941926 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.129:44415 (local address 127.15.113.130:55011)
0809 22:54:46.943727 (+ 1801us) negotiation.cc:107] Waiting for socket to connect
0809 22:54:46.943742 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 22:54:46.943874 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:54:46.944193 (+ 319us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:54:46.944196 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:54:46.944218 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:54:46.945278 (+ 1060us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:54:46.945292 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:54:46.946775 (+ 1483us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:54:46.946780 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:54:46.949533 (+ 2753us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:54:46.949546 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:54:46.951407 (+ 1861us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:54:46.951413 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:54:46.951532 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:54:46.951555 (+ 23us) client_negotiation.cc:769] Sending connection context
0809 22:54:46.951620 (+ 65us) client_negotiation.cc:240] Negotiation successful
0809 22:54:46.951702 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1616,"thread_start_us":1330,"threads_started":1}
I20260809 22:54:46.952667 1969 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "87dda3c64e5e43ce952537be7eaf149a" candidate_uuid: "106d3871856b48ebb6a2c8b60746ec2c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ceabead877694c22bd406c8f091718cf" is_pre_election: true
I20260809 22:54:46.953431 1558 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "87dda3c64e5e43ce952537be7eaf149a" candidate_uuid: "106d3871856b48ebb6a2c8b60746ec2c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8367c63aeaa441ba809773804a864e9f" is_pre_election: true
I20260809 22:54:46.953763 1969 raft_consensus.cc:2454] T 87dda3c64e5e43ce952537be7eaf149a P ceabead877694c22bd406c8f091718cf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 106d3871856b48ebb6a2c8b60746ec2c in term 0.
I20260809 22:54:46.954087 1558 raft_consensus.cc:2454] T 87dda3c64e5e43ce952537be7eaf149a P 8367c63aeaa441ba809773804a864e9f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 106d3871856b48ebb6a2c8b60746ec2c in term 0.
I20260809 22:54:46.954787 1679 leader_election.cc:302] T 87dda3c64e5e43ce952537be7eaf149a P 106d3871856b48ebb6a2c8b60746ec2c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 106d3871856b48ebb6a2c8b60746ec2c, ceabead877694c22bd406c8f091718cf; no voters:
I20260809 22:54:46.955335 2081 raft_consensus.cc:2781] T 87dda3c64e5e43ce952537be7eaf149a P 106d3871856b48ebb6a2c8b60746ec2c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 22:54:46.955610 2081 raft_consensus.cc:482] T 87dda3c64e5e43ce952537be7eaf149a P 106d3871856b48ebb6a2c8b60746ec2c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 22:54:46.955752 2081 raft_consensus.cc:3037] T 87dda3c64e5e43ce952537be7eaf149a P 106d3871856b48ebb6a2c8b60746ec2c [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:54:46.958724 2081 raft_consensus.cc:504] T 87dda3c64e5e43ce952537be7eaf149a P 106d3871856b48ebb6a2c8b60746ec2c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "106d3871856b48ebb6a2c8b60746ec2c" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 41727 } } peers { permanent_uuid: "8367c63aeaa441ba809773804a864e9f" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 44415 } } peers { permanent_uuid: "ceabead877694c22bd406c8f091718cf" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 45815 } }
I20260809 22:54:46.959378 2081 leader_election.cc:288] T 87dda3c64e5e43ce952537be7eaf149a P 106d3871856b48ebb6a2c8b60746ec2c [CANDIDATE]: Term 1 election: Requested vote from peers 8367c63aeaa441ba809773804a864e9f (127.15.113.129:44415), ceabead877694c22bd406c8f091718cf (127.15.113.131:45815)
I20260809 22:54:46.959764 1558 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "87dda3c64e5e43ce952537be7eaf149a" candidate_uuid: "106d3871856b48ebb6a2c8b60746ec2c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8367c63aeaa441ba809773804a864e9f"
I20260809 22:54:46.959951 1558 raft_consensus.cc:3037] T 87dda3c64e5e43ce952537be7eaf149a P 8367c63aeaa441ba809773804a864e9f [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:54:46.960127 1969 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "87dda3c64e5e43ce952537be7eaf149a" candidate_uuid: "106d3871856b48ebb6a2c8b60746ec2c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ceabead877694c22bd406c8f091718cf"
I20260809 22:54:46.960314 1969 raft_consensus.cc:3037] T 87dda3c64e5e43ce952537be7eaf149a P ceabead877694c22bd406c8f091718cf [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:54:46.963678 1558 raft_consensus.cc:2454] T 87dda3c64e5e43ce952537be7eaf149a P 8367c63aeaa441ba809773804a864e9f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 106d3871856b48ebb6a2c8b60746ec2c in term 1.
I20260809 22:54:46.963958 1969 raft_consensus.cc:2454] T 87dda3c64e5e43ce952537be7eaf149a P ceabead877694c22bd406c8f091718cf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 106d3871856b48ebb6a2c8b60746ec2c in term 1.
I20260809 22:54:46.964342 1675 leader_election.cc:302] T 87dda3c64e5e43ce952537be7eaf149a P 106d3871856b48ebb6a2c8b60746ec2c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 106d3871856b48ebb6a2c8b60746ec2c, 8367c63aeaa441ba809773804a864e9f; no voters:
I20260809 22:54:46.964650 2081 raft_consensus.cc:2781] T 87dda3c64e5e43ce952537be7eaf149a P 106d3871856b48ebb6a2c8b60746ec2c [term 1 FOLLOWER]: Leader election won for term 1
I20260809 22:54:46.965566 2081 raft_consensus.cc:686] T 87dda3c64e5e43ce952537be7eaf149a P 106d3871856b48ebb6a2c8b60746ec2c [term 1 LEADER]: Becoming Leader. State: Replica: 106d3871856b48ebb6a2c8b60746ec2c, State: Running, Role: LEADER
I20260809 22:54:46.966344 2081 consensus_queue.cc:237] T 87dda3c64e5e43ce952537be7eaf149a P 106d3871856b48ebb6a2c8b60746ec2c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "106d3871856b48ebb6a2c8b60746ec2c" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 41727 } } peers { permanent_uuid: "8367c63aeaa441ba809773804a864e9f" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 44415 } } peers { permanent_uuid: "ceabead877694c22bd406c8f091718cf" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 45815 } }
I20260809 22:54:46.972075 1315 catalog_manager.cc:5495] T 87dda3c64e5e43ce952537be7eaf149a P 106d3871856b48ebb6a2c8b60746ec2c reported cstate change: term changed from 0 to 1, leader changed from <none> to 106d3871856b48ebb6a2c8b60746ec2c (127.15.113.130). New cstate: current_term: 1 leader_uuid: "106d3871856b48ebb6a2c8b60746ec2c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "106d3871856b48ebb6a2c8b60746ec2c" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 41727 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8367c63aeaa441ba809773804a864e9f" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 44415 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ceabead877694c22bd406c8f091718cf" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 45815 } health_report { overall_health: UNKNOWN } } }
W20260809 22:54:46.988235 2037 tablet.cc:2369] T 87dda3c64e5e43ce952537be7eaf149a P ceabead877694c22bd406c8f091718cf: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 22:54:46.998310 2064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:54:46.991246 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45536 (local address 127.15.113.130:41727)
0809 22:54:46.991436 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0809 22:54:46.991441 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:54:46.991458 (+ 17us) server_negotiation.cc:407] Connection header received
0809 22:54:46.991634 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:54:46.991638 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:54:46.991753 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:54:46.991875 (+ 122us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 22:54:46.992356 (+ 481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:54:46.993445 (+ 1089us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:54:46.995852 (+ 2407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:54:46.996528 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:54:46.996624 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:54:46.997107 (+ 483us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 22:54:46.997690 (+ 583us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 22:54:46.997796 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0809 22:54:46.997891 (+ 95us) server_negotiation.cc:299] Negotiation successful
0809 22:54:46.997953 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260809 22:54:47.008914 1558 raft_consensus.cc:1261] T 87dda3c64e5e43ce952537be7eaf149a P 8367c63aeaa441ba809773804a864e9f [term 1 FOLLOWER]: Refusing update from remote peer 106d3871856b48ebb6a2c8b60746ec2c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 22:54:47.009244 1969 raft_consensus.cc:1261] T 87dda3c64e5e43ce952537be7eaf149a P ceabead877694c22bd406c8f091718cf [term 1 FOLLOWER]: Refusing update from remote peer 106d3871856b48ebb6a2c8b60746ec2c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 22:54:47.010100 2086 consensus_queue.cc:1035] T 87dda3c64e5e43ce952537be7eaf149a P 106d3871856b48ebb6a2c8b60746ec2c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8367c63aeaa441ba809773804a864e9f" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 44415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260809 22:54:47.010773 2081 consensus_queue.cc:1035] T 87dda3c64e5e43ce952537be7eaf149a P 106d3871856b48ebb6a2c8b60746ec2c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceabead877694c22bd406c8f091718cf" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 45815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 22:54:47.025314 2091 mvcc.cc:205] Tried to move back new op lower bound from 7316750692377309184 to 7316750692219211776. Current Snapshot: MvccSnapshot[applied={T|T < 7316750692377309184}]
I20260809 22:54:47.043816 1315 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55338:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 22:54:47.044305 1315 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55338:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 22:54:47.049067 1315 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c3b85d087ce245f48007e6c17e6f7303: Sending DeleteTablet for 3 replicas of tablet 87dda3c64e5e43ce952537be7eaf149a
I20260809 22:54:47.050616 1526 tablet_service.cc:1514] Processing DeleteTablet for tablet 87dda3c64e5e43ce952537be7eaf149a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:54:47 UTC) from {username='kudu', principal='kudu/127.15.113.190@KRBTEST.COM'} at 127.0.0.1:45552
I20260809 22:54:47.050832 1745 tablet_service.cc:1514] Processing DeleteTablet for tablet 87dda3c64e5e43ce952537be7eaf149a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:54:47 UTC) from {username='kudu', principal='kudu/127.15.113.190@KRBTEST.COM'} at 127.0.0.1:45522
I20260809 22:54:47.051999 1936 tablet_service.cc:1514] Processing DeleteTablet for tablet 87dda3c64e5e43ce952537be7eaf149a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:54:47 UTC) from {username='kudu', principal='kudu/127.15.113.190@KRBTEST.COM'} at 127.0.0.1:38840
I20260809 22:54:47.053406 15814 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskItYPJR/build/asan/bin/kudu pid 1433
I20260809 22:54:47.053548 2107 tablet_replica.cc:330] T 87dda3c64e5e43ce952537be7eaf149a P ceabead877694c22bd406c8f091718cf: stopping tablet replica
I20260809 22:54:47.054049 2107 raft_consensus.cc:2229] T 87dda3c64e5e43ce952537be7eaf149a P ceabead877694c22bd406c8f091718cf [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 22:54:47.054886 2107 raft_consensus.cc:2258] T 87dda3c64e5e43ce952537be7eaf149a P ceabead877694c22bd406c8f091718cf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 22:54:47.056495 2106 tablet_replica.cc:330] T 87dda3c64e5e43ce952537be7eaf149a P 8367c63aeaa441ba809773804a864e9f: stopping tablet replica
I20260809 22:54:47.057087 2106 raft_consensus.cc:2229] T 87dda3c64e5e43ce952537be7eaf149a P 8367c63aeaa441ba809773804a864e9f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 22:54:47.057801 2106 raft_consensus.cc:2258] T 87dda3c64e5e43ce952537be7eaf149a P 8367c63aeaa441ba809773804a864e9f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 22:54:47.058429 2108 tablet_replica.cc:330] T 87dda3c64e5e43ce952537be7eaf149a P 106d3871856b48ebb6a2c8b60746ec2c: stopping tablet replica
I20260809 22:54:47.058580 2107 ts_tablet_manager.cc:1905] T 87dda3c64e5e43ce952537be7eaf149a P ceabead877694c22bd406c8f091718cf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 22:54:47.060316 2106 ts_tablet_manager.cc:1905] T 87dda3c64e5e43ce952537be7eaf149a P 8367c63aeaa441ba809773804a864e9f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 22:54:47.059168 2108 raft_consensus.cc:2229] T 87dda3c64e5e43ce952537be7eaf149a P 106d3871856b48ebb6a2c8b60746ec2c [term 1 LEADER]: Raft consensus shutting down.
I20260809 22:54:47.067749 2106 ts_tablet_manager.cc:1918] T 87dda3c64e5e43ce952537be7eaf149a P 8367c63aeaa441ba809773804a864e9f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 22:54:47.068019 2106 log.cc:1196] T 87dda3c64e5e43ce952537be7eaf149a P 8367c63aeaa441ba809773804a864e9f: Deleting WAL directory at /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315935617919-15814-0/minicluster-data/ts-0/wal/wals/87dda3c64e5e43ce952537be7eaf149a
I20260809 22:54:47.068554 2107 ts_tablet_manager.cc:1918] T 87dda3c64e5e43ce952537be7eaf149a P ceabead877694c22bd406c8f091718cf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 22:54:47.068812 2107 log.cc:1196] T 87dda3c64e5e43ce952537be7eaf149a P ceabead877694c22bd406c8f091718cf: Deleting WAL directory at /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315935617919-15814-0/minicluster-data/ts-2/wal/wals/87dda3c64e5e43ce952537be7eaf149a
I20260809 22:54:47.068869 2106 ts_tablet_manager.cc:1939] T 87dda3c64e5e43ce952537be7eaf149a P 8367c63aeaa441ba809773804a864e9f: Deleting consensus metadata
Aug 09 22:54:47 dist-test-slave-ncp9 krb5kdc[567](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316081, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.113.129@KRBTEST.COM
I20260809 22:54:47.069728 2107 ts_tablet_manager.cc:1939] T 87dda3c64e5e43ce952537be7eaf149a P ceabead877694c22bd406c8f091718cf: Deleting consensus metadata
I20260809 22:54:47.071035 1295 catalog_manager.cc:4841] TS 8367c63aeaa441ba809773804a864e9f (127.15.113.129:44415): tablet 87dda3c64e5e43ce952537be7eaf149a (table test-table [id=fa3bab96f50d46808dd819895c2516d4]) successfully deleted
I20260809 22:54:47.071740 2108 raft_consensus.cc:2258] T 87dda3c64e5e43ce952537be7eaf149a P 106d3871856b48ebb6a2c8b60746ec2c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 22:54:47.071978 1296 catalog_manager.cc:4841] TS ceabead877694c22bd406c8f091718cf (127.15.113.131:45815): tablet 87dda3c64e5e43ce952537be7eaf149a (table test-table [id=fa3bab96f50d46808dd819895c2516d4]) successfully deleted
I20260809 22:54:47.074013 2108 ts_tablet_manager.cc:1905] T 87dda3c64e5e43ce952537be7eaf149a P 106d3871856b48ebb6a2c8b60746ec2c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 22:54:47.080590 2066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:54:47.054507 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45564 (local address 127.15.113.129:44415)
0809 22:54:47.054672 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0809 22:54:47.054678 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:54:47.057483 (+ 2805us) server_negotiation.cc:407] Connection header received
0809 22:54:47.057661 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:54:47.057665 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:54:47.057752 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:54:47.057859 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:54:47.059739 (+ 1880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:54:47.061274 (+ 1535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:54:47.063365 (+ 2091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:54:47.064131 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:54:47.064272 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:54:47.071313 (+ 7041us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:54:47.071327 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:54:47.071343 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:54:47.071382 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:54:47.076283 (+ 4901us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:54:47.077262 (+ 979us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:54:47.077271 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:54:47.077275 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:54:47.077500 (+ 225us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:54:47.078001 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:54:47.078006 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:54:47.078008 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:54:47.078735 (+ 727us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:54:47.079194 (+ 459us) server_negotiation.cc:1035] Waiting for connection context
0809 22:54:47.079401 (+ 207us) server_negotiation.cc:299] Negotiation successful
0809 22:54:47.079894 (+ 493us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260809 22:54:47.081410 1626 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:54:47.084160 2108 ts_tablet_manager.cc:1918] T 87dda3c64e5e43ce952537be7eaf149a P 106d3871856b48ebb6a2c8b60746ec2c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 22:54:47.084390 2108 log.cc:1196] T 87dda3c64e5e43ce952537be7eaf149a P 106d3871856b48ebb6a2c8b60746ec2c: Deleting WAL directory at /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315935617919-15814-0/minicluster-data/ts-1/wal/wals/87dda3c64e5e43ce952537be7eaf149a
I20260809 22:54:47.085038 2108 ts_tablet_manager.cc:1939] T 87dda3c64e5e43ce952537be7eaf149a P 106d3871856b48ebb6a2c8b60746ec2c: Deleting consensus metadata
I20260809 22:54:47.087075 1299 catalog_manager.cc:4841] TS 106d3871856b48ebb6a2c8b60746ec2c (127.15.113.130:41727): tablet 87dda3c64e5e43ce952537be7eaf149a (table test-table [id=fa3bab96f50d46808dd819895c2516d4]) successfully deleted
I20260809 22:54:47.174774 15814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskItYPJR/build/asan/bin/kudu with pid 1433
I20260809 22:54:47.205631 15814 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskItYPJR/build/asan/bin/kudu pid 1643
Aug 09 22:54:47 dist-test-slave-ncp9 krb5kdc[567](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316081, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.113.130@KRBTEST.COM
I20260809 22:54:47.223078 2064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:54:47.206354 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45544 (local address 127.15.113.130:41727)
0809 22:54:47.206554 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0809 22:54:47.206558 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 22:54:47.206605 (+ 47us) server_negotiation.cc:407] Connection header received
0809 22:54:47.206818 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:54:47.206822 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:54:47.206887 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:54:47.206966 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:54:47.208696 (+ 1730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:54:47.209583 (+ 887us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:54:47.211455 (+ 1872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:54:47.211938 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:54:47.212038 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:54:47.217142 (+ 5104us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:54:47.217156 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:54:47.217172 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:54:47.217211 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:54:47.220147 (+ 2936us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:54:47.221122 (+ 975us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:54:47.221129 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:54:47.221131 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:54:47.221291 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:54:47.221666 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:54:47.221670 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:54:47.221672 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:54:47.222040 (+ 368us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:54:47.222308 (+ 268us) server_negotiation.cc:1035] Waiting for connection context
0809 22:54:47.222649 (+ 341us) server_negotiation.cc:299] Negotiation successful
0809 22:54:47.222804 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260809 22:54:47.223901 1828 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:54:47.328869 15814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskItYPJR/build/asan/bin/kudu with pid 1643
I20260809 22:54:47.353410 15814 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskItYPJR/build/asan/bin/kudu pid 1843
Aug 09 22:54:47 dist-test-slave-ncp9 krb5kdc[567](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316081, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.113.131@KRBTEST.COM
I20260809 22:54:47.370746 2067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:54:47.354133 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38852 (local address 127.15.113.131:45815)
0809 22:54:47.354299 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0809 22:54:47.354303 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 22:54:47.354380 (+ 77us) server_negotiation.cc:407] Connection header received
0809 22:54:47.354523 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:54:47.354527 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:54:47.354599 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:54:47.354689 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:54:47.355934 (+ 1245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:54:47.356670 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:54:47.358262 (+ 1592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:54:47.358784 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:54:47.358874 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:54:47.365797 (+ 6923us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:54:47.365811 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:54:47.365829 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:54:47.365862 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:54:47.368335 (+ 2473us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:54:47.369093 (+ 758us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:54:47.369101 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:54:47.369103 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:54:47.369224 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:54:47.369527 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:54:47.369530 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:54:47.369531 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:54:47.369905 (+ 374us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:54:47.370112 (+ 207us) server_negotiation.cc:1035] Waiting for connection context
0809 22:54:47.370314 (+ 202us) server_negotiation.cc:299] Negotiation successful
0809 22:54:47.370473 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260809 22:54:47.371342 2029 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:54:47.458069 15814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskItYPJR/build/asan/bin/kudu with pid 1843
I20260809 22:54:47.481927 15814 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskItYPJR/build/asan/bin/kudu pid 1267
I20260809 22:54:47.483215 1362 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:54:47.594291 15814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskItYPJR/build/asan/bin/kudu with pid 1267
I20260809 22:54:47.623194 15814 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260809 22:54:47.966658 15814 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260809 22:54:47.966866 15814 mini_ranger.cc:66] Stopping Ranger...
I20260809 22:54:48.310283 15814 mini_ranger.cc:68] Stopped Ranger
2026-08-09 22:54:48.310 UTC [611] LOG: received smart shutdown request
2026-08-09 22:54:48.313 UTC [611] LOG: background worker "logical replication launcher" (PID 621) exited with exit code 1
2026-08-09 22:54:48.314 UTC [615] LOG: shutting down
2026-08-09 22:54:48.421 UTC [611] LOG: database system is shut down
2026-08-09T22:54:48Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (49866 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315935617919-15814-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 22:54:48 dist-test-slave-ncp9 krb5kdc[2126](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 22:54:48 dist-test-slave-ncp9 krb5kdc[2126](info): set up 2 sockets
Aug 09 22:54:48 dist-test-slave-ncp9 krb5kdc[2126](info): commencing operation
krb5kdc: starting...
W20260809 22:54:49.756265 15814 mini_kdc.cc:121] Time spent starting KDC: real 1.191s user 0.000s sys 0.035s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315935617919-15814-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315935617919-15814-0/krb5kdc/test-user.keytab.
Aug 09 22:54:49 dist-test-slave-ncp9 krb5kdc[2126](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316089, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T22:54:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T22:54:49Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.15.113.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.15.113.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.15.113.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315935617919-15814-0/krb5kdc/rangeradmin_127.15.113.148@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.15.113.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315935617919-15814-0/krb5kdc/rangeradmin_127.15.113.148@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.15.113.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.15.113.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.15.113.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315935617919-15814-0/krb5kdc/rangerlookup_127.15.113.148@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.15.113.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315935617919-15814-0/krb5kdc/rangerlookup_127.15.113.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.15.113.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.15.113.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.15.113.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315935617919-15814-0/krb5kdc/HTTP_127.15.113.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.15.113.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315935617919-15814-0/krb5kdc/HTTP_127.15.113.148@KRBTEST.COM.keytab.
I20260809 22:54:50.088475 15814 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315935617919-15814-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskItYPJR/build/asan/bin/postgres/pg_ctl -D /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315935617919-15814-0/minicluster-data/postgres -l logfile start
2026-08-09 22:54:52.326 UTC [2170] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-09 22:54:52.326 UTC [2170] LOG: listening on IPv4 address "127.15.113.148", port 54587
2026-08-09 22:54:52.329 UTC [2170] LOG: listening on Unix socket "/tmp/.s.PGSQL.54587"
2026-08-09 22:54:52.350 UTC [2178] LOG: database system was shut down at 2026-08-09 22:54:51 UTC
2026-08-09 22:54:52.353 UTC [2170] LOG: database system is ready to accept connections
I20260809 22:54:53.433535 15814 mini_postgres.cc:96] Postgres bound to 54587
2026-08-09 22:54:53.458 UTC [2196] FATAL: database "slave" does not exist
127.15.113.148:54587 - accepting connections
I20260809 22:54:53.459293 15814 mini_ranger.cc:161] Starting Ranger...
I20260809 22:54:53.487190 15814 mini_ranger.cc:84] Created miniranger Postgres user
I20260809 22:54:53.903091 15814 mini_ranger.cc:87] Created ranger Postgres database
I20260809 22:54:53.903198 15814 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315935617919-15814-0/minicluster-data/ranger-admin
2026-08-09 22:54:54,171 [I] DB FLAVOR :POSTGRES
2026-08-09 22:54:54,171 [I] --------- Verifying Ranger DB connection ---------
2026-08-09 22:54:54,172 [I] Checking connection..
2026-08-09 22:54:54,172 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskItYPJR/build/asan/bin/postgresql.jar:/tmp/dist-test-taskItYPJR/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.113.148:54587/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-09 22:54:54,426 [I] Checking connection passed.
2026-08-09 22:54:54,427 [I] --------- Verifying version history table ---------
2026-08-09 22:54:54,427 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskItYPJR/build/asan/bin/postgresql.jar:/tmp/dist-test-taskItYPJR/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.113.148:54587/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-09 22:54:54,674 [I] Table x_db_version_h does not exist in database ranger
2026-08-09 22:54:54,674 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskItYPJR/build/asan/bin/postgresql.jar:/tmp/dist-test-taskItYPJR/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.113.148:54587/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-09 22:54:54,909 [I] Table x_db_version_h does not exist in database ranger
2026-08-09 22:54:54,909 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-09 22:54:54,909 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskItYPJR/build/asan/bin/postgresql.jar:/tmp/dist-test-taskItYPJR/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.113.148:54587/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskItYPJR/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-09 22:54:55.160 UTC [2289] WARNING: there is no transaction in progress
2026-08-09 22:54:55,168 [I] create_dbversion_catalog.sql file imported successfully
2026-08-09 22:54:55,168 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskItYPJR/build/asan/bin/postgresql.jar:/tmp/dist-test-taskItYPJR/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.113.148:54587/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-09 22:54:55,406 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-09 22:54:55,406 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-09 22:54:55,406 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskItYPJR/build/asan/bin/postgresql.jar:/tmp/dist-test-taskItYPJR/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.113.148:54587/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-09 22:54:55,638 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskItYPJR/build/asan/bin/postgresql.jar:/tmp/dist-test-taskItYPJR/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.113.148:54587/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-09 22:54:55,875 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskItYPJR/build/asan/bin/postgresql.jar:/tmp/dist-test-taskItYPJR/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.113.148:54587/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-ncp9.c.gcp-upstream.internal','N') ;"
2026-08-09 22:54:56,100 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskItYPJR/build/asan/bin/postgresql.jar:/tmp/dist-test-taskItYPJR/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.113.148:54587/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-09 22:54:56,351 [I] Table x_portal_user does not exist in database ranger
2026-08-09 22:54:56,351 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskItYPJR/build/asan/bin/postgresql.jar:/tmp/dist-test-taskItYPJR/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.113.148:54587/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-09 22:54:56,595 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-09 22:54:56,596 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskItYPJR/build/asan/bin/postgresql.jar:/tmp/dist-test-taskItYPJR/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.113.148:54587/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-09 22:54:56,852 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskItYPJR/build/asan/bin/postgresql.jar:/tmp/dist-test-taskItYPJR/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.113.148:54587/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-09 22:54:57,107 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-09 22:54:57,108 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskItYPJR/build/asan/bin/postgresql.jar:/tmp/dist-test-taskItYPJR/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.113.148:54587/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskItYPJR/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-09 22:54:57.978 UTC [2461] WARNING: there is no transaction in progress
2026-08-09 22:54:57.993 UTC [2461] WARNING: there is no transaction in progress
2026-08-09 22:54:58.008 UTC [2461] WARNING: there is no transaction in progress
2026-08-09 22:54:58.022 UTC [2461] WARNING: there is no transaction in progress
2026-08-09 22:54:58.035 UTC [2461] WARNING: there is no transaction in progress
2026-08-09 22:54:58.052 UTC [2461] WARNING: there is no transaction in progress
2026-08-09 22:54:58.101 UTC [2461] WARNING: there is no transaction in progress
2026-08-09 22:54:58.107 UTC [2461] WARNING: there is no transaction in progress
2026-08-09 22:54:58.120 UTC [2461] WARNING: there is no transaction in progress
2026-08-09 22:54:58.129 UTC [2461] WARNING: there is no transaction in progress
2026-08-09 22:54:58.138 UTC [2461] WARNING: there is no transaction in progress
2026-08-09 22:54:58.151 UTC [2461] WARNING: there is no transaction in progress
2026-08-09 22:54:58.159 UTC [2461] WARNING: there is no transaction in progress
2026-08-09 22:54:58.164 UTC [2461] WARNING: there is no transaction in progress
2026-08-09 22:54:58.170 UTC [2461] WARNING: there is no transaction in progress
2026-08-09 22:54:58.175 UTC [2461] WARNING: there is no transaction in progress
2026-08-09 22:54:58,714 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-09 22:54:58,714 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskItYPJR/build/asan/bin/postgresql.jar:/tmp/dist-test-taskItYPJR/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.113.148:54587/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-09 22:54:58,960 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-09 22:54:58,961 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskItYPJR/build/asan/bin/postgresql.jar:/tmp/dist-test-taskItYPJR/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.113.148:54587/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-09 22:54:59,218 [I] Table x_portal_user already exists in database 'ranger'
2026-08-09 22:54:59,218 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskItYPJR/build/asan/bin/postgresql.jar:/tmp/dist-test-taskItYPJR/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.113.148:54587/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-09 22:54:59,472 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-09 22:54:59,472 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskItYPJR/build/asan/bin/postgresql.jar:/tmp/dist-test-taskItYPJR/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.113.148:54587/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-09 22:54:59,721 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskItYPJR/build/asan/bin/postgresql.jar:/tmp/dist-test-taskItYPJR/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.113.148:54587/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-09 22:54:59,971 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskItYPJR/build/asan/bin/postgresql.jar:/tmp/dist-test-taskItYPJR/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.113.148:54587/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-ncp9.c.gcp-upstream.internal';"
2026-08-09 22:55:00,221 [I] CORE_DB_SCHEMA import status has been updated
2026-08-09 22:55:00,221 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskItYPJR/build/asan/bin/postgresql.jar:/tmp/dist-test-taskItYPJR/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.113.148:54587/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-09 22:55:00,467 [I] DB_PATCHES have already been applied
I20260809 22:55:00.473289 15814 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315935617919-15814-0/minicluster-data/ranger-admin:/tmp/dist-test-taskItYPJR/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskItYPJR/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskItYPJR/build/asan/bin/postgresql.jar:/tmp/dist-test-taskItYPJR/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260809 22:55:00.473376 15814 mini_ranger.cc:193] Using host: 127.15.113.148
I20260809 22:55:00.493579 15814 mini_ranger.cc:227] Ranger admin URL: http://127.15.113.148:55545
Aug 09, 2026 10:55:00 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 09, 2026 10:55:00 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 09, 2026 10:55:00 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315935617919-15814-0/minicluster-data/ranger-admin/ews/webapp
Aug 09, 2026 10:55:00 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315935617919-15814-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 09, 2026 10:55:00 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315935617919-15814-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 09, 2026 10:55:01 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315935617919-15814-0/minicluster-data/ranger-admin/ews/webapp].
Aug 09, 2026 10:55:01 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260809 22:55:01.727659 15814 mini_ranger.cc:160] Time spent starting Ranger: real 8.268s user 0.001s sys 0.055s
Aug 09, 2026 10:55:21 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 09, 2026 10:55:21 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 09, 2026 10:55:21 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 09, 2026 10:55:21 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 09, 2026 10:55:21 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 09, 2026 10:55:21 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 09, 2026 10:55:21 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 09, 2026 10:55:21 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 09, 2026 10:55:21 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 09, 2026 10:55:21 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 09, 2026 10:55:21 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 09, 2026 10:55:21 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 09, 2026 10:55:21 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 09, 2026 10:55:21 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 09, 2026 10:55:21 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 09, 2026 10:55:21 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 09, 2026 10:55:21 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 09, 2026 10:55:22 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260809 22:55:22.738090 15814 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.15.113.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.15.113.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.15.113.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315935617919-15814-0/krb5kdc/rangerkms_127.15.113.148@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.15.113.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315935617919-15814-0/krb5kdc/rangerkms_127.15.113.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.15.113.148@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.15.113.148@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.15.113.148@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315935617919-15814-0/krb5kdc/HTTP_127.15.113.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.15.113.148@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315935617919-15814-0/krb5kdc/HTTP_127.15.113.148@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 09 22:55:22 dist-test-slave-ncp9 krb5kdc[2126](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316122, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260809 22:55:22.922070 15814 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260809 22:55:22.947361 15814 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260809 22:55:23.677435 15814 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260809 22:55:23.677527 15814 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315935617919-15814-0/minicluster-data/ranger-kms
I20260809 22:55:23.677556 15814 mini_ranger_kms.cc:227] Using postgres at 127.15.113.148:54587
2026-08-09 22:55:24,012 [I] DB FLAVOR :POSTGRES
2026-08-09 22:55:24,012 [I] --------- Verifying Ranger DB connection ---------
2026-08-09 22:55:24,012 [I] Checking connection
2026-08-09 22:55:24,012 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskItYPJR/build/asan/bin/postgresql.jar:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315935617919-15814-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.113.148:54587/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-09 22:55:24,261 [I] connection success
2026-08-09 22:55:24,261 [I] --------- Verifying Ranger DB tables ---------
2026-08-09 22:55:24,261 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-09 22:55:24,261 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskItYPJR/build/asan/bin/postgresql.jar:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315935617919-15814-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.113.148:54587/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-09 22:55:24,486 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-09 22:55:24,487 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-09 22:55:24,487 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-09 22:55:24,487 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskItYPJR/build/asan/bin/postgresql.jar:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315935617919-15814-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.113.148:54587/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315935617919-15814-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-09 22:55:24,745 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260809 22:55:25.152397 15814 mini_ranger_kms.cc:310] Created kms service
I20260809 22:55:25.342554 15814 mini_ranger_kms.cc:326] Created kudu user
I20260809 22:55:25.384845 15814 mini_ranger_kms.cc:343] Created rangerkms user
Aug 09 22:55:25 dist-test-slave-ncp9 krb5kdc[2126](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316122, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.15.113.148@KRBTEST.COM
I20260809 22:55:25.767644 15814 mini_ranger_kms.cc:384] Added ranger policy
I20260809 22:55:25.768000 15814 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315935617919-15814-0/minicluster-data/ranger-kms:/tmp/dist-test-taskItYPJR/build/asan/bin/postgresql.jar:/tmp/dist-test-taskItYPJR/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskItYPJR/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260809 22:55:25.768065 15814 mini_ranger_kms.cc:240] Using host: 127.15.113.148
I20260809 22:55:25.788718 15814 mini_ranger_kms.cc:276] Ranger KMS PID: 2745
I20260809 22:55:25.788844 15814 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.15.113.148:47973
Aug 09, 2026 10:55:26 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 09, 2026 10:55:26 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 09, 2026 10:55:26 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315935617919-15814-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 09, 2026 10:55:26 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315935617919-15814-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 09, 2026 10:55:27 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315935617919-15814-0/minicluster-data/ranger-kms/ews/webapp].
Aug 09, 2026 10:55:27 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 09 22:55:28 dist-test-slave-ncp9 krb5kdc[2126](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786316128, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.15.113.148@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:55:28.278286 15814 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.356s user 0.058s sys 0.310s
I20260809 22:55:28.278560 15814 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260809 22:55:28.278741 15814 mini_ranger_kms.cc:401] 127.15.113.148:47973/kms/v1/keys
Aug 09 22:55:28 dist-test-slave-ncp9 krb5kdc[2126](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316128, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.15.113.148@KRBTEST.COM for HTTP/127.15.113.148@KRBTEST.COM
Aug 09 22:55:28 dist-test-slave-ncp9 krb5kdc[2126](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316128, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.15.113.148@KRBTEST.COM for HTTP/127.15.113.148@KRBTEST.COM
Aug 09 22:55:31 dist-test-slave-ncp9 krb5kdc[2126](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316131, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.15.113.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.113.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.113.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.190.keytab.
Entry for principal kudu/127.15.113.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.190.keytab.
I20260809 22:55:31.209702 15814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskItYPJR/build/asan/bin/kudu
/tmp/dist-test-taskItYPJR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315935617919-15814-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315935617919-15814-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315935617919-15814-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315935617919-15814-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.113.190:34121
--webserver_interface=127.15.113.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.190.keytab
--principal=kudu/127.15.113.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.113.148:40529
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.113.190:34121
--ranger_config_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315935617919-15814-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.15.113.148:47973/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315935617919-15814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315935617919-15814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315935617919-15814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:55:31.528331 2828 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:55:31.528627 2828 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:55:31.528769 2828 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:55:31.535853 2828 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 22:55:31.535972 2828 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:55:31.536038 2828 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:55:31.536114 2828 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 22:55:31.536180 2828 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 22:55:31.546054 2828 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.113.148:40529
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315935617919-15814-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315935617919-15814-0/minicluster-data/master-0/wal
--ranger_kms_url=127.15.113.148:47973/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.15.113.190:34121
--ranger_config_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315935617919-15814-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.190.keytab
--principal=kudu/127.15.113.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.113.190:34121
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315935617919-15814-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.113.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315935617919-15814-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:55:31.547302 2828 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:55:31.548581 2828 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:55:31.560178 2836 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:55:31.560251 2837 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:55:31.560215 2839 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:55:31.560847 2828 server_base.cc:1034] running on GCE node
I20260809 22:55:31.561580 2828 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:55:31.562865 2828 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:55:31.564055 2828 hybrid_clock.cc:648] HybridClock initialized: now 1786316131564010 us; error 35 us; skew 500 ppm
I20260809 22:55:31.564368 2828 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:55:31 dist-test-slave-ncp9 krb5kdc[2126](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316131, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.113.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:55:31.567308 2828 init.cc:377] Logged in from keytab as kudu/127.15.113.190@KRBTEST.COM (short username kudu)
I20260809 22:55:31.568593 2828 webserver.cc:460] Webserver started at http://127.15.113.190:39179/ using document root <none> and password file <none>
I20260809 22:55:31.569235 2828 fs_manager.cc:359] Metadata directory not provided
I20260809 22:55:31.569334 2828 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:55:31.569581 2828 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 22:55:31 dist-test-slave-ncp9 krb5kdc[2126](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316131, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.113.190@KRBTEST.COM for HTTP/127.15.113.148@KRBTEST.COM
I20260809 22:55:31.634261 2828 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315935617919-15814-0/minicluster-data/master-0/data/instance:
uuid: "b38b4bc8428a47cea24947c1d328666f"
format_stamp: "Formatted at 2026-08-09 22:55:31 on dist-test-slave-ncp9"
server_key: "9c6488eb29fc8392233654dcd7675325"
server_key_iv: "1d719f967dd0c9fcf447f73c083e790f"
server_key_version: "kuduclusterkey@0"
I20260809 22:55:31.635100 2828 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315935617919-15814-0/minicluster-data/master-0/wal/instance:
uuid: "b38b4bc8428a47cea24947c1d328666f"
format_stamp: "Formatted at 2026-08-09 22:55:31 on dist-test-slave-ncp9"
server_key: "9c6488eb29fc8392233654dcd7675325"
server_key_iv: "1d719f967dd0c9fcf447f73c083e790f"
server_key_version: "kuduclusterkey@0"
I20260809 22:55:31.640842 2828 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260809 22:55:31.658532 2853 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:55:31.659955 2828 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 22:55:31.660138 2828 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315935617919-15814-0/minicluster-data/master-0/data,/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315935617919-15814-0/minicluster-data/master-0/wal
uuid: "b38b4bc8428a47cea24947c1d328666f"
format_stamp: "Formatted at 2026-08-09 22:55:31 on dist-test-slave-ncp9"
server_key: "9c6488eb29fc8392233654dcd7675325"
server_key_iv: "1d719f967dd0c9fcf447f73c083e790f"
server_key_version: "kuduclusterkey@0"
I20260809 22:55:31.660348 2828 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315935617919-15814-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315935617919-15814-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315935617919-15814-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:55:31.684443 2828 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:55:31.688287 2828 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:55:31.688591 2828 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:55:31.708909 2828 rpc_server.cc:307] RPC server started. Bound to: 127.15.113.190:34121
I20260809 22:55:31.708920 2929 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.113.190:34121 every 8 connection(s)
I20260809 22:55:31.710246 2828 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315935617919-15814-0/minicluster-data/master-0/data/info.pb
I20260809 22:55:31.712466 15814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskItYPJR/build/asan/bin/kudu as pid 2828
I20260809 22:55:31.712702 15814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315935617919-15814-0/minicluster-data/master-0/wal/instance
I20260809 22:55:31.713564 15814 external_mini_cluster.cc:1428] Setting key b64ea2c103d6a9b8091c7ef6fd4d790f
I20260809 22:55:31.714318 2931 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:55:31.730863 2931 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b38b4bc8428a47cea24947c1d328666f: Bootstrap starting.
I20260809 22:55:31.734095 2931 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b38b4bc8428a47cea24947c1d328666f: Neither blocks nor log segments found. Creating new log.
I20260809 22:55:31.735319 2931 log.cc:824] T 00000000000000000000000000000000 P b38b4bc8428a47cea24947c1d328666f: Log is configured to *not* fsync() on all Append() calls
I20260809 22:55:31.738229 2931 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b38b4bc8428a47cea24947c1d328666f: No bootstrap required, opened a new log
Aug 09 22:55:31 dist-test-slave-ncp9 krb5kdc[2126](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316131, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.113.190@KRBTEST.COM
I20260809 22:55:31.746908 2931 raft_consensus.cc:348] T 00000000000000000000000000000000 P b38b4bc8428a47cea24947c1d328666f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b38b4bc8428a47cea24947c1d328666f" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 34121 } }
I20260809 22:55:31.747269 2931 raft_consensus.cc:374] T 00000000000000000000000000000000 P b38b4bc8428a47cea24947c1d328666f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:55:31.747382 2931 raft_consensus.cc:729] T 00000000000000000000000000000000 P b38b4bc8428a47cea24947c1d328666f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b38b4bc8428a47cea24947c1d328666f, State: Initialized, Role: FOLLOWER
I20260809 22:55:31.748021 2931 consensus_queue.cc:260] T 00000000000000000000000000000000 P b38b4bc8428a47cea24947c1d328666f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b38b4bc8428a47cea24947c1d328666f" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 34121 } }
I20260809 22:55:31.748277 2931 raft_consensus.cc:388] T 00000000000000000000000000000000 P b38b4bc8428a47cea24947c1d328666f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 22:55:31.748418 2931 raft_consensus.cc:482] T 00000000000000000000000000000000 P b38b4bc8428a47cea24947c1d328666f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 22:55:31.748622 2931 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b38b4bc8428a47cea24947c1d328666f [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:55:31.751021 2935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:55:31.715684 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58588 (local address 127.15.113.190:34121)
0809 22:55:31.729393 (+ 13709us) server_negotiation.cc:206] Beginning negotiation
0809 22:55:31.729405 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 22:55:31.729903 (+ 498us) server_negotiation.cc:407] Connection header received
0809 22:55:31.732449 (+ 2546us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:55:31.732480 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:55:31.733013 (+ 533us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:55:31.733434 (+ 421us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:55:31.735043 (+ 1609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:55:31.737132 (+ 2089us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:55:31.738905 (+ 1773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:55:31.739559 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:55:31.739684 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:55:31.744283 (+ 4599us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:55:31.744290 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:55:31.744305 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:55:31.744341 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:55:31.747152 (+ 2811us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:55:31.747932 (+ 780us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:55:31.747938 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:55:31.747944 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:55:31.748092 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:55:31.748500 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:55:31.748504 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:55:31.748507 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:55:31.749097 (+ 590us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:55:31.749329 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0809 22:55:31.749726 (+ 397us) server_negotiation.cc:299] Negotiation successful
0809 22:55:31.750048 (+ 322us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13495,"thread_start_us":13267,"threads_started":1}
I20260809 22:55:31.751206 2931 raft_consensus.cc:504] T 00000000000000000000000000000000 P b38b4bc8428a47cea24947c1d328666f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b38b4bc8428a47cea24947c1d328666f" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 34121 } }
I20260809 22:55:31.751711 2931 leader_election.cc:302] T 00000000000000000000000000000000 P b38b4bc8428a47cea24947c1d328666f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b38b4bc8428a47cea24947c1d328666f; no voters:
I20260809 22:55:31.752301 2931 leader_election.cc:288] T 00000000000000000000000000000000 P b38b4bc8428a47cea24947c1d328666f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 22:55:31.752390 2937 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b38b4bc8428a47cea24947c1d328666f [term 1 FOLLOWER]: Leader election won for term 1
I20260809 22:55:31.754268 2937 raft_consensus.cc:686] T 00000000000000000000000000000000 P b38b4bc8428a47cea24947c1d328666f [term 1 LEADER]: Becoming Leader. State: Replica: b38b4bc8428a47cea24947c1d328666f, State: Running, Role: LEADER
I20260809 22:55:31.754606 2931 sys_catalog.cc:564] T 00000000000000000000000000000000 P b38b4bc8428a47cea24947c1d328666f [sys.catalog]: configured and running, proceeding with master startup.
I20260809 22:55:31.754869 2937 consensus_queue.cc:237] T 00000000000000000000000000000000 P b38b4bc8428a47cea24947c1d328666f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b38b4bc8428a47cea24947c1d328666f" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 34121 } }
I20260809 22:55:31.760682 2938 sys_catalog.cc:455] T 00000000000000000000000000000000 P b38b4bc8428a47cea24947c1d328666f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b38b4bc8428a47cea24947c1d328666f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b38b4bc8428a47cea24947c1d328666f" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 34121 } } }
I20260809 22:55:31.760748 2939 sys_catalog.cc:455] T 00000000000000000000000000000000 P b38b4bc8428a47cea24947c1d328666f [sys.catalog]: SysCatalogTable state changed. Reason: New leader b38b4bc8428a47cea24947c1d328666f. Latest consensus state: current_term: 1 leader_uuid: "b38b4bc8428a47cea24947c1d328666f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b38b4bc8428a47cea24947c1d328666f" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 34121 } } }
I20260809 22:55:31.761087 2938 sys_catalog.cc:458] T 00000000000000000000000000000000 P b38b4bc8428a47cea24947c1d328666f [sys.catalog]: This master's current role is: LEADER
I20260809 22:55:31.761132 2939 sys_catalog.cc:458] T 00000000000000000000000000000000 P b38b4bc8428a47cea24947c1d328666f [sys.catalog]: This master's current role is: LEADER
I20260809 22:55:31.762557 2951 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 22:55:31.768146 2951 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 22:55:31.768875 2931 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315935617919-15814-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260809 22:55:31.787346 2951 catalog_manager.cc:1347] Generated new cluster ID: 935a852950604cdb91e2e644081a3b10
I20260809 22:55:31.787480 2951 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 22:55:31.813444 2951 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 22:55:31.814785 2951 catalog_manager.cc:1504] Loading token signing keys...
I20260809 22:55:31.826143 2951 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b38b4bc8428a47cea24947c1d328666f: Generated new TSK 0
I20260809 22:55:31.826930 2951 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.113.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.113.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.113.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.129.keytab.
Entry for principal kudu/127.15.113.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.129.keytab.
I20260809 22:55:32.588256 15814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskItYPJR/build/asan/bin/kudu
/tmp/dist-test-taskItYPJR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315935617919-15814-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315935617919-15814-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315935617919-15814-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315935617919-15814-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.15.113.148:47973/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.113.129:0
--local_ip_for_outbound_sockets=127.15.113.129
--webserver_interface=127.15.113.129
--webserver_port=0
--tserver_master_addrs=127.15.113.190:34121
--keytab_file=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.129.keytab
--principal=kudu/127.15.113.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.113.148:40529
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315935617919-15814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315935617919-15814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315935617919-15814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 22:55:32 dist-test-slave-ncp9 krb5kdc[2126](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786316132, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.113.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260809 22:55:32.964866 3007 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:55:32.965214 3007 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:55:32.965364 3007 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:55:32.972800 3007 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:55:32.973006 3007 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:55:32.973171 3007 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.113.129
I20260809 22:55:32.984225 3007 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.113.148:40529
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315935617919-15814-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315935617919-15814-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.15.113.148:47973/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.129.keytab
--principal=kudu/127.15.113.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.113.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315935617919-15814-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.113.129
--webserver_port=0
--tserver_master_addrs=127.15.113.190:34121
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.113.129
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315935617919-15814-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:55:32.985560 3007 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:55:32.987111 3007 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:55:33.005049 3021 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:55:33.005045 3020 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:55:33.016703 3023 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:55:33.017402 3007 server_base.cc:1034] running on GCE node
I20260809 22:55:33.018205 3007 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:55:33.020133 3007 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:55:33.021425 3007 hybrid_clock.cc:648] HybridClock initialized: now 1786316133021315 us; error 91 us; skew 500 ppm
I20260809 22:55:33.021833 3007 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:55:33 dist-test-slave-ncp9 krb5kdc[2126](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316133, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.113.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:55:33.025938 3007 init.cc:377] Logged in from keytab as kudu/127.15.113.129@KRBTEST.COM (short username kudu)
I20260809 22:55:33.027771 3007 webserver.cc:460] Webserver started at http://127.15.113.129:39167/ using document root <none> and password file <none>
I20260809 22:55:33.028724 3007 fs_manager.cc:359] Metadata directory not provided
I20260809 22:55:33.028873 3007 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:55:33.029377 3007 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 22:55:33 dist-test-slave-ncp9 krb5kdc[2126](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316133, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.113.129@KRBTEST.COM for HTTP/127.15.113.148@KRBTEST.COM
I20260809 22:55:33.048509 3007 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315935617919-15814-0/minicluster-data/ts-0/data/instance:
uuid: "1e145cb790454107aeda329005bdc491"
format_stamp: "Formatted at 2026-08-09 22:55:33 on dist-test-slave-ncp9"
server_key: "2efc1a6cc97f01c4221634866cb191b6"
server_key_iv: "d4eac8ae6817a82614020933cccda6e4"
server_key_version: "kuduclusterkey@0"
I20260809 22:55:33.049475 3007 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315935617919-15814-0/minicluster-data/ts-0/wal/instance:
uuid: "1e145cb790454107aeda329005bdc491"
format_stamp: "Formatted at 2026-08-09 22:55:33 on dist-test-slave-ncp9"
server_key: "2efc1a6cc97f01c4221634866cb191b6"
server_key_iv: "d4eac8ae6817a82614020933cccda6e4"
server_key_version: "kuduclusterkey@0"
I20260809 22:55:33.056520 3007 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.006s sys 0.000s
I20260809 22:55:33.073751 3031 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:55:33.075628 3007 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260809 22:55:33.075914 3007 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315935617919-15814-0/minicluster-data/ts-0/data,/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315935617919-15814-0/minicluster-data/ts-0/wal
uuid: "1e145cb790454107aeda329005bdc491"
format_stamp: "Formatted at 2026-08-09 22:55:33 on dist-test-slave-ncp9"
server_key: "2efc1a6cc97f01c4221634866cb191b6"
server_key_iv: "d4eac8ae6817a82614020933cccda6e4"
server_key_version: "kuduclusterkey@0"
I20260809 22:55:33.076259 3007 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315935617919-15814-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315935617919-15814-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315935617919-15814-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:55:33.100759 3007 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:55:33.105953 3007 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:55:33.106349 3007 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:55:33.113536 3007 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:55:33.115025 3007 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:55:33.115109 3007 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:55:33.115264 3007 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:55:33.115324 3007 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:55:33.173749 3007 rpc_server.cc:307] RPC server started. Bound to: 127.15.113.129:34291
I20260809 22:55:33.173759 3189 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.113.129:34291 every 8 connection(s)
I20260809 22:55:33.175614 3007 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315935617919-15814-0/minicluster-data/ts-0/data/info.pb
I20260809 22:55:33.183818 15814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskItYPJR/build/asan/bin/kudu as pid 3007
I20260809 22:55:33.184024 15814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315935617919-15814-0/minicluster-data/ts-0/wal/instance
I20260809 22:55:33.184842 15814 external_mini_cluster.cc:1428] Setting key 04d63046e3552bee083c1eac469bbb9c
WARNING: no policy specified for kudu/127.15.113.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.113.130@KRBTEST.COM" created.
Aug 09 22:55:33 dist-test-slave-ncp9 krb5kdc[2126](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316133, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.113.129@KRBTEST.COM for kudu/127.15.113.190@KRBTEST.COM
I20260809 22:55:33.237330 3194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:55:33.191380 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.113.129:39307 (local address 127.15.113.190:34121)
0809 22:55:33.201025 (+ 9645us) server_negotiation.cc:206] Beginning negotiation
0809 22:55:33.201031 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:55:33.207355 (+ 6324us) server_negotiation.cc:407] Connection header received
0809 22:55:33.212032 (+ 4677us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:55:33.212038 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:55:33.212134 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:55:33.212210 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:55:33.220425 (+ 8215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:55:33.221672 (+ 1247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:55:33.224273 (+ 2601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:55:33.224969 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:55:33.225075 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:55:33.229789 (+ 4714us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:55:33.229796 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:55:33.229802 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:55:33.229845 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:55:33.233360 (+ 3515us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:55:33.234400 (+ 1040us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:55:33.234406 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:55:33.234410 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:55:33.234579 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:55:33.235125 (+ 546us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:55:33.235131 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:55:33.235135 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:55:33.235516 (+ 381us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:55:33.235642 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0809 22:55:33.236676 (+ 1034us) server_negotiation.cc:299] Negotiation successful
0809 22:55:33.236928 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9496,"thread_start_us":9380,"threads_started":1}
I20260809 22:55:33.239949 3195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:55:33.192017 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:34121 (local address 127.15.113.129:39307)
0809 22:55:33.207170 (+ 15153us) negotiation.cc:107] Waiting for socket to connect
0809 22:55:33.207227 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0809 22:55:33.211642 (+ 4415us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:55:33.217781 (+ 6139us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:55:33.217795 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:55:33.219267 (+ 1472us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:55:33.220179 (+ 912us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:55:33.220212 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:55:33.221888 (+ 1676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:55:33.221894 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:55:33.224022 (+ 2128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:55:33.224039 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:55:33.225471 (+ 1432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:55:33.225481 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:55:33.225786 (+ 305us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:55:33.226579 (+ 793us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:55:33.226612 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:55:33.229507 (+ 2895us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:55:33.233628 (+ 4121us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:55:33.233639 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:55:33.233657 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:55:33.234254 (+ 597us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:55:33.234731 (+ 477us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:55:33.234736 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:55:33.234739 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:55:33.234989 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:55:33.235748 (+ 759us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:55:33.235756 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:55:33.236196 (+ 440us) client_negotiation.cc:769] Sending connection context
0809 22:55:33.237907 (+ 1711us) client_negotiation.cc:240] Negotiation successful
0809 22:55:33.238483 (+ 576us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14833,"thread_start_us":15472,"threads_started":1}
I20260809 22:55:33.241912 3190 heartbeater.cc:347] Connected to a master server at 127.15.113.190:34121
I20260809 22:55:33.242281 3190 heartbeater.cc:464] Registering TS with master...
I20260809 22:55:33.243155 3190 heartbeater.cc:511] Master 127.15.113.190:34121 requested a full tablet report, sending...
I20260809 22:55:33.245618 2881 ts_manager.cc:194] Registered new tserver with Master: 1e145cb790454107aeda329005bdc491 (127.15.113.129:34291)
I20260809 22:55:33.247538 2881 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.113.129@KRBTEST.COM'} at 127.15.113.129:39307
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.113.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.130.keytab.
Entry for principal kudu/127.15.113.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.130.keytab.
I20260809 22:55:33.268380 15814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskItYPJR/build/asan/bin/kudu
/tmp/dist-test-taskItYPJR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315935617919-15814-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315935617919-15814-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315935617919-15814-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315935617919-15814-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.15.113.148:47973/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.113.130:0
--local_ip_for_outbound_sockets=127.15.113.130
--webserver_interface=127.15.113.130
--webserver_port=0
--tserver_master_addrs=127.15.113.190:34121
--keytab_file=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.130.keytab
--principal=kudu/127.15.113.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.113.148:40529
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315935617919-15814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315935617919-15814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315935617919-15814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 22:55:33 dist-test-slave-ncp9 krb5kdc[2126](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316132, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.113.190@KRBTEST.COM for HTTP/127.15.113.148@KRBTEST.COM
Aug 09 22:55:33 dist-test-slave-ncp9 krb5kdc[2126](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316132, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.113.190@KRBTEST.COM for HTTP/127.15.113.148@KRBTEST.COM
W20260809 22:55:33.748718 3199 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:55:33.749141 3199 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:55:33.749301 3199 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:55:33.760603 3199 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:55:33.760784 3199 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:55:33.761029 3199 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.113.130
I20260809 22:55:33.776437 3199 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.113.148:40529
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315935617919-15814-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315935617919-15814-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.15.113.148:47973/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.130.keytab
--principal=kudu/127.15.113.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.113.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315935617919-15814-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.113.130
--webserver_port=0
--tserver_master_addrs=127.15.113.190:34121
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.113.130
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315935617919-15814-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:55:33.778342 3199 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:55:33.780071 3199 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:55:33.833771 3212 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:55:33.840636 3199 server_base.cc:1034] running on GCE node
W20260809 22:55:33.841404 3210 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:55:33.847365 3218 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:55:33.848234 3199 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:55:33.851900 3199 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:55:33.853169 3199 hybrid_clock.cc:648] HybridClock initialized: now 1786316133853067 us; error 91 us; skew 500 ppm
I20260809 22:55:33.853601 3199 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:55:33 dist-test-slave-ncp9 krb5kdc[2126](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316133, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.113.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:55:33.857884 3199 init.cc:377] Logged in from keytab as kudu/127.15.113.130@KRBTEST.COM (short username kudu)
I20260809 22:55:33.878320 3199 webserver.cc:460] Webserver started at http://127.15.113.130:38683/ using document root <none> and password file <none>
I20260809 22:55:33.879274 3199 fs_manager.cc:359] Metadata directory not provided
I20260809 22:55:33.879415 3199 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:55:33.879828 3199 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 22:55:33 dist-test-slave-ncp9 krb5kdc[2126](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316133, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.113.130@KRBTEST.COM for HTTP/127.15.113.148@KRBTEST.COM
I20260809 22:55:33.910907 3199 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315935617919-15814-0/minicluster-data/ts-1/data/instance:
uuid: "a619374402094f629bb96421437ea9aa"
format_stamp: "Formatted at 2026-08-09 22:55:33 on dist-test-slave-ncp9"
server_key: "62a69f8afb08edb69dfa35b8043cff73"
server_key_iv: "dc7899767d9d16b361ccf8670c43b0d1"
server_key_version: "kuduclusterkey@0"
I20260809 22:55:33.912626 3199 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315935617919-15814-0/minicluster-data/ts-1/wal/instance:
uuid: "a619374402094f629bb96421437ea9aa"
format_stamp: "Formatted at 2026-08-09 22:55:33 on dist-test-slave-ncp9"
server_key: "62a69f8afb08edb69dfa35b8043cff73"
server_key_iv: "dc7899767d9d16b361ccf8670c43b0d1"
server_key_version: "kuduclusterkey@0"
I20260809 22:55:33.925999 3199 fs_manager.cc:691] Time spent creating directory manager: real 0.013s user 0.011s sys 0.000s
I20260809 22:55:33.972715 3226 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:55:33.982740 3199 fs_manager.cc:725] Time spent opening block manager: real 0.025s user 0.004s sys 0.002s
I20260809 22:55:33.983088 3199 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315935617919-15814-0/minicluster-data/ts-1/data,/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315935617919-15814-0/minicluster-data/ts-1/wal
uuid: "a619374402094f629bb96421437ea9aa"
format_stamp: "Formatted at 2026-08-09 22:55:33 on dist-test-slave-ncp9"
server_key: "62a69f8afb08edb69dfa35b8043cff73"
server_key_iv: "dc7899767d9d16b361ccf8670c43b0d1"
server_key_version: "kuduclusterkey@0"
I20260809 22:55:33.983471 3199 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315935617919-15814-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315935617919-15814-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315935617919-15814-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:55:34.045349 3199 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:55:34.050947 3199 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:55:34.051362 3199 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:55:34.055528 3199 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:55:34.065331 3199 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:55:34.065574 3199 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:55:34.065776 3199 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:55:34.065886 3199 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:55:34.122529 3199 rpc_server.cc:307] RPC server started. Bound to: 127.15.113.130:43941
I20260809 22:55:34.122556 3412 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.113.130:43941 every 8 connection(s)
I20260809 22:55:34.124369 3199 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315935617919-15814-0/minicluster-data/ts-1/data/info.pb
I20260809 22:55:34.129892 15814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskItYPJR/build/asan/bin/kudu as pid 3199
I20260809 22:55:34.130051 15814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315935617919-15814-0/minicluster-data/ts-1/wal/instance
I20260809 22:55:34.130609 15814 external_mini_cluster.cc:1428] Setting key 488cb5a0d122c79cb7d01f922e16d559
Aug 09 22:55:34 dist-test-slave-ncp9 krb5kdc[2126](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316133, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.113.130@KRBTEST.COM for kudu/127.15.113.190@KRBTEST.COM
I20260809 22:55:34.167418 3418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:55:34.128524 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.113.130:35011 (local address 127.15.113.190:34121)
0809 22:55:34.130514 (+ 1990us) server_negotiation.cc:206] Beginning negotiation
0809 22:55:34.130520 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:55:34.142032 (+ 11512us) server_negotiation.cc:407] Connection header received
0809 22:55:34.144157 (+ 2125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:55:34.144163 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:55:34.144257 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:55:34.144374 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:55:34.149294 (+ 4920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:55:34.150476 (+ 1182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:55:34.152804 (+ 2328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:55:34.153464 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:55:34.153592 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:55:34.159250 (+ 5658us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:55:34.159257 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:55:34.159262 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:55:34.159304 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:55:34.162876 (+ 3572us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:55:34.164735 (+ 1859us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:55:34.164741 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:55:34.164745 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:55:34.164913 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:55:34.165439 (+ 526us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:55:34.165444 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:55:34.165447 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:55:34.165819 (+ 372us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:55:34.165954 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0809 22:55:34.166798 (+ 844us) server_negotiation.cc:299] Negotiation successful
0809 22:55:34.167032 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1856,"thread_start_us":1745,"threads_started":1}
I20260809 22:55:34.168716 3417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:55:34.127894 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:34121 (local address 127.15.113.130:35011)
0809 22:55:34.141482 (+ 13588us) negotiation.cc:107] Waiting for socket to connect
0809 22:55:34.141536 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0809 22:55:34.143847 (+ 2311us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:55:34.147567 (+ 3720us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:55:34.147580 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:55:34.148214 (+ 634us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:55:34.149075 (+ 861us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:55:34.149104 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:55:34.150676 (+ 1572us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:55:34.150681 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:55:34.152627 (+ 1946us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:55:34.152642 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:55:34.153883 (+ 1241us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:55:34.153890 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:55:34.154172 (+ 282us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:55:34.154917 (+ 745us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:55:34.154947 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:55:34.159031 (+ 4084us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:55:34.163076 (+ 4045us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:55:34.163086 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:55:34.163104 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:55:34.163671 (+ 567us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:55:34.165023 (+ 1352us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:55:34.165028 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:55:34.165032 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:55:34.165333 (+ 301us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:55:34.165966 (+ 633us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:55:34.165975 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:55:34.166383 (+ 408us) client_negotiation.cc:769] Sending connection context
0809 22:55:34.166683 (+ 300us) client_negotiation.cc:240] Negotiation successful
0809 22:55:34.167191 (+ 508us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13265,"thread_start_us":13042,"threads_started":1}
WARNING: no policy specified for kudu/127.15.113.131@KRBTEST.COM; defaulting to no policy
I20260809 22:55:34.170593 3414 heartbeater.cc:347] Connected to a master server at 127.15.113.190:34121
I20260809 22:55:34.170899 3414 heartbeater.cc:464] Registering TS with master...
I20260809 22:55:34.171725 3414 heartbeater.cc:511] Master 127.15.113.190:34121 requested a full tablet report, sending...
I20260809 22:55:34.177383 2881 ts_manager.cc:194] Registered new tserver with Master: a619374402094f629bb96421437ea9aa (127.15.113.130:43941)
I20260809 22:55:34.178439 2881 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.113.130@KRBTEST.COM'} at 127.15.113.130:35011
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.113.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.113.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.131.keytab.
Entry for principal kudu/127.15.113.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.131.keytab.
I20260809 22:55:34.222465 15814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskItYPJR/build/asan/bin/kudu
/tmp/dist-test-taskItYPJR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315935617919-15814-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315935617919-15814-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315935617919-15814-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315935617919-15814-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.15.113.148:47973/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.113.131:0
--local_ip_for_outbound_sockets=127.15.113.131
--webserver_interface=127.15.113.131
--webserver_port=0
--tserver_master_addrs=127.15.113.190:34121
--keytab_file=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.131.keytab
--principal=kudu/127.15.113.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.113.148:40529
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315935617919-15814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315935617919-15814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315935617919-15814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 22:55:34.251024 3190 heartbeater.cc:503] Master 127.15.113.190:34121 was elected leader, sending a full tablet report...
W20260809 22:55:34.602448 3423 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:55:34.602722 3423 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:55:34.602874 3423 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:55:34.609994 3423 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:55:34.610116 3423 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:55:34.610271 3423 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.113.131
I20260809 22:55:34.619863 3423 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.113.148:40529
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315935617919-15814-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315935617919-15814-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.15.113.148:47973/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.131.keytab
--principal=kudu/127.15.113.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.113.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315935617919-15814-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.113.131
--webserver_port=0
--tserver_master_addrs=127.15.113.190:34121
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.113.131
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315935617919-15814-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:55:34.621165 3423 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:55:34.622493 3423 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:55:34.635064 3432 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:55:34.635049 3433 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:55:34.635102 3435 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:55:34.635753 3423 server_base.cc:1034] running on GCE node
I20260809 22:55:34.636267 3423 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:55:34.637148 3423 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:55:34.638350 3423 hybrid_clock.cc:648] HybridClock initialized: now 1786316134638268 us; error 70 us; skew 500 ppm
I20260809 22:55:34.638640 3423 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:55:34 dist-test-slave-ncp9 krb5kdc[2126](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316134, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.113.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:55:34.641669 3423 init.cc:377] Logged in from keytab as kudu/127.15.113.131@KRBTEST.COM (short username kudu)
I20260809 22:55:34.642944 3423 webserver.cc:460] Webserver started at http://127.15.113.131:40959/ using document root <none> and password file <none>
I20260809 22:55:34.643558 3423 fs_manager.cc:359] Metadata directory not provided
I20260809 22:55:34.643652 3423 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:55:34.643895 3423 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 22:55:34 dist-test-slave-ncp9 krb5kdc[2126](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316134, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.113.131@KRBTEST.COM for HTTP/127.15.113.148@KRBTEST.COM
I20260809 22:55:34.658231 3423 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315935617919-15814-0/minicluster-data/ts-2/data/instance:
uuid: "6e9036e451a84c2aa357fa45343e4c7c"
format_stamp: "Formatted at 2026-08-09 22:55:34 on dist-test-slave-ncp9"
server_key: "0e5fef32080dd1eda8374921de1513f8"
server_key_iv: "5a49346f815a5959a264bc9ce2d79960"
server_key_version: "kuduclusterkey@0"
I20260809 22:55:34.659318 3423 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315935617919-15814-0/minicluster-data/ts-2/wal/instance:
uuid: "6e9036e451a84c2aa357fa45343e4c7c"
format_stamp: "Formatted at 2026-08-09 22:55:34 on dist-test-slave-ncp9"
server_key: "0e5fef32080dd1eda8374921de1513f8"
server_key_iv: "5a49346f815a5959a264bc9ce2d79960"
server_key_version: "kuduclusterkey@0"
I20260809 22:55:34.665376 3423 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.001s
I20260809 22:55:34.680862 3443 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:55:34.682405 3423 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 22:55:34.682554 3423 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315935617919-15814-0/minicluster-data/ts-2/data,/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315935617919-15814-0/minicluster-data/ts-2/wal
uuid: "6e9036e451a84c2aa357fa45343e4c7c"
format_stamp: "Formatted at 2026-08-09 22:55:34 on dist-test-slave-ncp9"
server_key: "0e5fef32080dd1eda8374921de1513f8"
server_key_iv: "5a49346f815a5959a264bc9ce2d79960"
server_key_version: "kuduclusterkey@0"
I20260809 22:55:34.682739 3423 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315935617919-15814-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315935617919-15814-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315935617919-15814-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:55:34.694823 3423 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:55:34.698645 3423 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:55:34.698909 3423 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:55:34.699812 3423 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:55:34.701217 3423 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:55:34.701295 3423 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:55:34.701411 3423 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:55:34.701462 3423 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:55:34.737371 3423 rpc_server.cc:307] RPC server started. Bound to: 127.15.113.131:36419
I20260809 22:55:34.737426 3617 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.113.131:36419 every 8 connection(s)
I20260809 22:55:34.738638 3423 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315935617919-15814-0/minicluster-data/ts-2/data/info.pb
I20260809 22:55:34.746969 15814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskItYPJR/build/asan/bin/kudu as pid 3423
I20260809 22:55:34.747134 15814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315935617919-15814-0/minicluster-data/ts-2/wal/instance
I20260809 22:55:34.747588 15814 external_mini_cluster.cc:1428] Setting key 2475c5182227fbc7821d630bf43f39d2
Aug 09 22:55:34 dist-test-slave-ncp9 krb5kdc[2126](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316134, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.113.131@KRBTEST.COM for kudu/127.15.113.190@KRBTEST.COM
I20260809 22:55:34.757526 3620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:55:34.741172 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.113.131:55175 (local address 127.15.113.190:34121)
0809 22:55:34.741655 (+ 483us) server_negotiation.cc:206] Beginning negotiation
0809 22:55:34.741659 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 22:55:34.742745 (+ 1086us) server_negotiation.cc:407] Connection header received
0809 22:55:34.744390 (+ 1645us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:55:34.744394 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:55:34.744477 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:55:34.744570 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:55:34.745942 (+ 1372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:55:34.746789 (+ 847us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:55:34.748597 (+ 1808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:55:34.749145 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:55:34.749274 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:55:34.752432 (+ 3158us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:55:34.752437 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:55:34.752441 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:55:34.752477 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:55:34.754954 (+ 2477us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:55:34.755571 (+ 617us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:55:34.755575 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:55:34.755577 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:55:34.755699 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:55:34.756129 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:55:34.756133 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:55:34.756135 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:55:34.756381 (+ 246us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:55:34.756497 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0809 22:55:34.757094 (+ 597us) server_negotiation.cc:299] Negotiation successful
0809 22:55:34.757249 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":374,"thread_start_us":226,"threads_started":1}
I20260809 22:55:34.758327 3621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:55:34.741811 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:34121 (local address 127.15.113.131:55175)
0809 22:55:34.742578 (+ 767us) negotiation.cc:107] Waiting for socket to connect
0809 22:55:34.742632 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0809 22:55:34.744159 (+ 1527us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:55:34.744762 (+ 603us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:55:34.744769 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:55:34.745199 (+ 430us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:55:34.745785 (+ 586us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:55:34.745804 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:55:34.746981 (+ 1177us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:55:34.746992 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:55:34.748462 (+ 1470us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:55:34.748477 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:55:34.749365 (+ 888us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:55:34.749373 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:55:34.749628 (+ 255us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:55:34.750304 (+ 676us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:55:34.750332 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:55:34.752237 (+ 1905us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:55:34.755072 (+ 2835us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:55:34.755078 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:55:34.755088 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:55:34.755464 (+ 376us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:55:34.755783 (+ 319us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:55:34.755786 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:55:34.755788 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:55:34.756033 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:55:34.756476 (+ 443us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:55:34.756482 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:55:34.756753 (+ 271us) client_negotiation.cc:769] Sending connection context
0809 22:55:34.757014 (+ 261us) client_negotiation.cc:240] Negotiation successful
0809 22:55:34.757358 (+ 344us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":519,"thread_start_us":292,"threads_started":1}
I20260809 22:55:34.759613 3618 heartbeater.cc:347] Connected to a master server at 127.15.113.190:34121
I20260809 22:55:34.759857 3618 heartbeater.cc:464] Registering TS with master...
I20260809 22:55:34.760519 3618 heartbeater.cc:511] Master 127.15.113.190:34121 requested a full tablet report, sending...
I20260809 22:55:34.761940 2881 ts_manager.cc:194] Registered new tserver with Master: 6e9036e451a84c2aa357fa45343e4c7c (127.15.113.131:36419)
I20260809 22:55:34.762980 15814 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 22:55:34.762952 2881 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.113.131@KRBTEST.COM'} at 127.15.113.131:55175
I20260809 22:55:34.764873 15814 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskItYPJR/build/asan/bin/kudu pid 3007
Aug 09 22:55:34 dist-test-slave-ncp9 krb5kdc[2126](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316131, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.113.129@KRBTEST.COM
I20260809 22:55:34.779829 3624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:55:34.765818 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54400 (local address 127.15.113.129:34291)
0809 22:55:34.766396 (+ 578us) server_negotiation.cc:206] Beginning negotiation
0809 22:55:34.766402 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:55:34.766424 (+ 22us) server_negotiation.cc:407] Connection header received
0809 22:55:34.766478 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:55:34.766485 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:55:34.766645 (+ 160us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:55:34.766759 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:55:34.768103 (+ 1344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:55:34.769119 (+ 1016us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:55:34.770644 (+ 1525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:55:34.771149 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:55:34.771239 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:55:34.775074 (+ 3835us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:55:34.775084 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:55:34.775096 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:55:34.775126 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:55:34.777582 (+ 2456us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:55:34.778189 (+ 607us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:55:34.778195 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:55:34.778197 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:55:34.778323 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:55:34.778632 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:55:34.778635 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:55:34.778637 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:55:34.778924 (+ 287us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:55:34.779139 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0809 22:55:34.779336 (+ 197us) server_negotiation.cc:299] Negotiation successful
0809 22:55:34.779510 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":444,"thread_start_us":303,"threads_started":1}
I20260809 22:55:34.781044 3185 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:55:34.860653 15814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskItYPJR/build/asan/bin/kudu with pid 3007
I20260809 22:55:34.880260 15814 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskItYPJR/build/asan/bin/kudu pid 3199
Aug 09 22:55:34 dist-test-slave-ncp9 krb5kdc[2126](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316131, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.113.130@KRBTEST.COM
I20260809 22:55:34.895627 3626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:55:34.881101 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36634 (local address 127.15.113.130:43941)
0809 22:55:34.881605 (+ 504us) server_negotiation.cc:206] Beginning negotiation
0809 22:55:34.881612 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:55:34.881638 (+ 26us) server_negotiation.cc:407] Connection header received
0809 22:55:34.881699 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:55:34.881706 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:55:34.881904 (+ 198us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:55:34.882047 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:55:34.883150 (+ 1103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:55:34.884124 (+ 974us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:55:34.885553 (+ 1429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:55:34.886135 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:55:34.886226 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:55:34.890503 (+ 4277us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:55:34.890513 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:55:34.890527 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:55:34.890564 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:55:34.893206 (+ 2642us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:55:34.893884 (+ 678us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:55:34.893890 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:55:34.893893 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:55:34.894034 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:55:34.894350 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:55:34.894353 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:55:34.894356 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:55:34.894692 (+ 336us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:55:34.894906 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0809 22:55:34.895124 (+ 218us) server_negotiation.cc:299] Negotiation successful
0809 22:55:34.895322 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":358,"thread_start_us":246,"threads_started":1}
I20260809 22:55:34.896827 3407 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:55:34.976013 15814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskItYPJR/build/asan/bin/kudu with pid 3199
I20260809 22:55:34.996176 15814 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskItYPJR/build/asan/bin/kudu pid 3423
Aug 09 22:55:35 dist-test-slave-ncp9 krb5kdc[2126](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316131, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.113.131@KRBTEST.COM
I20260809 22:55:35.016553 3629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:55:35.001389 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43570 (local address 127.15.113.131:36419)
0809 22:55:35.002022 (+ 633us) server_negotiation.cc:206] Beginning negotiation
0809 22:55:35.002029 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:55:35.002056 (+ 27us) server_negotiation.cc:407] Connection header received
0809 22:55:35.002118 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:55:35.002125 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:55:35.002374 (+ 249us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:55:35.002538 (+ 164us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:55:35.003868 (+ 1330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:55:35.004879 (+ 1011us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:55:35.006180 (+ 1301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:55:35.006740 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:55:35.006852 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:55:35.011046 (+ 4194us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:55:35.011056 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:55:35.011070 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:55:35.011106 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:55:35.013696 (+ 2590us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:55:35.014342 (+ 646us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:55:35.014349 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:55:35.014351 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:55:35.014481 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:55:35.014851 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:55:35.014855 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:55:35.014857 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:55:35.015183 (+ 326us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:55:35.015480 (+ 297us) server_negotiation.cc:1035] Waiting for connection context
0809 22:55:35.015927 (+ 447us) server_negotiation.cc:299] Negotiation successful
0809 22:55:35.016202 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":478,"thread_start_us":325,"threads_started":1}
I20260809 22:55:35.017856 3610 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:55:35.097886 15814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskItYPJR/build/asan/bin/kudu with pid 3423
I20260809 22:55:35.121474 15814 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskItYPJR/build/asan/bin/kudu pid 2828
I20260809 22:55:35.122705 2924 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:55:35.226179 15814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskItYPJR/build/asan/bin/kudu with pid 2828
I20260809 22:55:35.253697 15814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskItYPJR/build/asan/bin/kudu
/tmp/dist-test-taskItYPJR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315935617919-15814-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315935617919-15814-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315935617919-15814-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315935617919-15814-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.113.190:34121
--webserver_interface=127.15.113.190
--webserver_port=39179
--keytab_file=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.190.keytab
--principal=kudu/127.15.113.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.113.148:40529
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.113.190:34121
--ranger_config_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315935617919-15814-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.15.113.148:47973/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.15.113.148:47973/kms with env {KRB5CCNAME=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315935617919-15814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315935617919-15814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315935617919-15814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:55:35.578795 3635 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:55:35.579087 3635 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:55:35.579234 3635 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:55:35.586259 3635 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 22:55:35.586369 3635 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:55:35.586443 3635 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:55:35.586520 3635 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 22:55:35.586585 3635 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 22:55:35.596228 3635 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.113.148:40529
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315935617919-15814-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315935617919-15814-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.15.113.148:47973/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.15.113.190:34121
--ranger_config_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315935617919-15814-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.190.keytab
--principal=kudu/127.15.113.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.113.190:34121
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315935617919-15814-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.113.190
--webserver_port=39179
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315935617919-15814-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:55:35.597496 3635 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:55:35.598783 3635 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:55:35.609854 3643 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:55:35.609912 3644 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:55:35.609931 3646 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:55:35.610806 3635 server_base.cc:1034] running on GCE node
I20260809 22:55:35.611436 3635 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:55:35.612665 3635 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:55:35.613862 3635 hybrid_clock.cc:648] HybridClock initialized: now 1786316135613811 us; error 38 us; skew 500 ppm
I20260809 22:55:35.614189 3635 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:55:35 dist-test-slave-ncp9 krb5kdc[2126](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316135, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.113.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:55:35.621003 3635 init.cc:377] Logged in from keytab as kudu/127.15.113.190@KRBTEST.COM (short username kudu)
I20260809 22:55:35.622506 3635 webserver.cc:460] Webserver started at http://127.15.113.190:39179/ using document root <none> and password file <none>
I20260809 22:55:35.623158 3635 fs_manager.cc:359] Metadata directory not provided
I20260809 22:55:35.623286 3635 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 09 22:55:35 dist-test-slave-ncp9 krb5kdc[2126](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316135, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.113.190@KRBTEST.COM for HTTP/127.15.113.148@KRBTEST.COM
I20260809 22:55:35.694708 3635 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.000s sys 0.004s
I20260809 22:55:35.697775 3664 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:55:35.699122 3635 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 22:55:35.699303 3635 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315935617919-15814-0/minicluster-data/master-0/data,/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315935617919-15814-0/minicluster-data/master-0/wal
uuid: "b38b4bc8428a47cea24947c1d328666f"
format_stamp: "Formatted at 2026-08-09 22:55:31 on dist-test-slave-ncp9"
server_key: "9c6488eb29fc8392233654dcd7675325"
server_key_iv: "1d719f967dd0c9fcf447f73c083e790f"
server_key_version: "kuduclusterkey@0"
I20260809 22:55:35.700253 3635 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315935617919-15814-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315935617919-15814-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315935617919-15814-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:55:35.716240 3635 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:55:35.719632 3635 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:55:35.719899 3635 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:55:35.736869 3635 rpc_server.cc:307] RPC server started. Bound to: 127.15.113.190:34121
I20260809 22:55:35.736910 3739 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.113.190:34121 every 8 connection(s)
I20260809 22:55:35.738298 3635 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315935617919-15814-0/minicluster-data/master-0/data/info.pb
I20260809 22:55:35.743168 15814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskItYPJR/build/asan/bin/kudu as pid 3635
I20260809 22:55:35.744213 15814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskItYPJR/build/asan/bin/kudu
/tmp/dist-test-taskItYPJR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315935617919-15814-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315935617919-15814-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315935617919-15814-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315935617919-15814-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.15.113.148:47973/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.113.129:34291
--local_ip_for_outbound_sockets=127.15.113.129
--tserver_master_addrs=127.15.113.190:34121
--webserver_port=39167
--webserver_interface=127.15.113.129
--keytab_file=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.129.keytab
--principal=kudu/127.15.113.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.113.148:40529
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.15.113.148:47973/kms with env {KRB5CCNAME=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315935617919-15814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315935617919-15814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315935617919-15814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 22:55:35.745857 3740 sys_catalog.cc:263] Verifying existing consensus state
I20260809 22:55:35.750083 3740 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b38b4bc8428a47cea24947c1d328666f: Bootstrap starting.
I20260809 22:55:35.770150 3740 log.cc:824] T 00000000000000000000000000000000 P b38b4bc8428a47cea24947c1d328666f: Log is configured to *not* fsync() on all Append() calls
I20260809 22:55:35.778708 3740 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b38b4bc8428a47cea24947c1d328666f: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 22:55:35.779188 3740 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b38b4bc8428a47cea24947c1d328666f: Bootstrap complete.
I20260809 22:55:35.787830 3740 raft_consensus.cc:348] T 00000000000000000000000000000000 P b38b4bc8428a47cea24947c1d328666f [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b38b4bc8428a47cea24947c1d328666f" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 34121 } }
I20260809 22:55:35.788626 3740 raft_consensus.cc:729] T 00000000000000000000000000000000 P b38b4bc8428a47cea24947c1d328666f [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: b38b4bc8428a47cea24947c1d328666f, State: Initialized, Role: FOLLOWER
I20260809 22:55:35.789234 3740 consensus_queue.cc:260] T 00000000000000000000000000000000 P b38b4bc8428a47cea24947c1d328666f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b38b4bc8428a47cea24947c1d328666f" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 34121 } }
I20260809 22:55:35.789443 3740 raft_consensus.cc:388] T 00000000000000000000000000000000 P b38b4bc8428a47cea24947c1d328666f [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 22:55:35.789554 3740 raft_consensus.cc:482] T 00000000000000000000000000000000 P b38b4bc8428a47cea24947c1d328666f [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 22:55:35.789733 3740 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b38b4bc8428a47cea24947c1d328666f [term 1 FOLLOWER]: Advancing to term 2
I20260809 22:55:35.793406 3740 raft_consensus.cc:504] T 00000000000000000000000000000000 P b38b4bc8428a47cea24947c1d328666f [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b38b4bc8428a47cea24947c1d328666f" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 34121 } }
I20260809 22:55:35.793866 3740 leader_election.cc:302] T 00000000000000000000000000000000 P b38b4bc8428a47cea24947c1d328666f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b38b4bc8428a47cea24947c1d328666f; no voters:
I20260809 22:55:35.794426 3740 leader_election.cc:288] T 00000000000000000000000000000000 P b38b4bc8428a47cea24947c1d328666f [CANDIDATE]: Term 2 election: Requested vote from peers
I20260809 22:55:35.794579 3745 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b38b4bc8428a47cea24947c1d328666f [term 2 FOLLOWER]: Leader election won for term 2
I20260809 22:55:35.795614 3745 raft_consensus.cc:686] T 00000000000000000000000000000000 P b38b4bc8428a47cea24947c1d328666f [term 2 LEADER]: Becoming Leader. State: Replica: b38b4bc8428a47cea24947c1d328666f, State: Running, Role: LEADER
I20260809 22:55:35.796212 3745 consensus_queue.cc:237] T 00000000000000000000000000000000 P b38b4bc8428a47cea24947c1d328666f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b38b4bc8428a47cea24947c1d328666f" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 34121 } }
I20260809 22:55:35.796466 3740 sys_catalog.cc:564] T 00000000000000000000000000000000 P b38b4bc8428a47cea24947c1d328666f [sys.catalog]: configured and running, proceeding with master startup.
I20260809 22:55:35.798794 3751 sys_catalog.cc:455] T 00000000000000000000000000000000 P b38b4bc8428a47cea24947c1d328666f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "b38b4bc8428a47cea24947c1d328666f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b38b4bc8428a47cea24947c1d328666f" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 34121 } } }
I20260809 22:55:35.799160 3751 sys_catalog.cc:458] T 00000000000000000000000000000000 P b38b4bc8428a47cea24947c1d328666f [sys.catalog]: This master's current role is: LEADER
I20260809 22:55:35.801699 3753 sys_catalog.cc:455] T 00000000000000000000000000000000 P b38b4bc8428a47cea24947c1d328666f [sys.catalog]: SysCatalogTable state changed. Reason: New leader b38b4bc8428a47cea24947c1d328666f. Latest consensus state: current_term: 2 leader_uuid: "b38b4bc8428a47cea24947c1d328666f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b38b4bc8428a47cea24947c1d328666f" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 34121 } } }
I20260809 22:55:35.802033 3753 sys_catalog.cc:458] T 00000000000000000000000000000000 P b38b4bc8428a47cea24947c1d328666f [sys.catalog]: This master's current role is: LEADER
I20260809 22:55:35.803475 3758 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 22:55:35.809144 3758 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 22:55:35.810505 3758 catalog_manager.cc:1259] Loaded cluster ID: 935a852950604cdb91e2e644081a3b10
I20260809 22:55:35.810644 3758 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 22:55:35.814602 3758 catalog_manager.cc:1504] Loading token signing keys...
I20260809 22:55:35.815982 3758 catalog_manager.cc:5879] T 00000000000000000000000000000000 P b38b4bc8428a47cea24947c1d328666f: Loaded TSK: 0
I20260809 22:55:35.817566 3758 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 22:55:35.820633 3740 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315935617919-15814-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
W20260809 22:55:36.202497 3743 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:55:36.202997 3743 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:55:36.203212 3743 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:55:36.214555 3743 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:55:36.214700 3743 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:55:36.214916 3743 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.113.129
I20260809 22:55:36.230280 3743 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.113.148:40529
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315935617919-15814-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315935617919-15814-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.15.113.148:47973/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.129.keytab
--principal=kudu/127.15.113.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.113.129:34291
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315935617919-15814-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.113.129
--webserver_port=39167
--tserver_master_addrs=127.15.113.190:34121
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.113.129
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315935617919-15814-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:55:36.232385 3743 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:55:36.234345 3743 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:55:36.298290 3797 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:55:36.298532 3796 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:55:36.304834 3800 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:55:36.306001 3743 server_base.cc:1034] running on GCE node
I20260809 22:55:36.306845 3743 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:55:36.308179 3743 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:55:36.309499 3743 hybrid_clock.cc:648] HybridClock initialized: now 1786316136309406 us; error 74 us; skew 500 ppm
I20260809 22:55:36.309952 3743 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:55:36 dist-test-slave-ncp9 krb5kdc[2126](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316136, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.113.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:55:36.316166 3743 init.cc:377] Logged in from keytab as kudu/127.15.113.129@KRBTEST.COM (short username kudu)
I20260809 22:55:36.338864 3743 webserver.cc:460] Webserver started at http://127.15.113.129:39167/ using document root <none> and password file <none>
I20260809 22:55:36.339896 3743 fs_manager.cc:359] Metadata directory not provided
I20260809 22:55:36.340147 3743 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 09 22:55:36 dist-test-slave-ncp9 krb5kdc[2126](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316136, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.113.129@KRBTEST.COM for HTTP/127.15.113.148@KRBTEST.COM
I20260809 22:55:36.383589 3743 fs_manager.cc:709] Time spent opening directory manager: real 0.006s user 0.004s sys 0.001s
I20260809 22:55:36.387002 3811 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:55:36.388589 3743 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260809 22:55:36.388762 3743 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315935617919-15814-0/minicluster-data/ts-0/data,/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315935617919-15814-0/minicluster-data/ts-0/wal
uuid: "1e145cb790454107aeda329005bdc491"
format_stamp: "Formatted at 2026-08-09 22:55:33 on dist-test-slave-ncp9"
server_key: "2efc1a6cc97f01c4221634866cb191b6"
server_key_iv: "d4eac8ae6817a82614020933cccda6e4"
server_key_version: "kuduclusterkey@0"
I20260809 22:55:36.390211 3743 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315935617919-15814-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315935617919-15814-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315935617919-15814-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:55:36.426231 3743 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:55:36.430424 3743 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:55:36.430728 3743 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:55:36.434211 3743 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:55:36.437947 3743 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:55:36.438058 3743 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:55:36.438170 3743 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:55:36.438249 3743 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:55:36.496986 3743 rpc_server.cc:307] RPC server started. Bound to: 127.15.113.129:34291
I20260809 22:55:36.497022 3980 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.113.129:34291 every 8 connection(s)
I20260809 22:55:36.498731 3743 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315935617919-15814-0/minicluster-data/ts-0/data/info.pb
I20260809 22:55:36.508255 15814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskItYPJR/build/asan/bin/kudu as pid 3743
I20260809 22:55:36.509183 15814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskItYPJR/build/asan/bin/kudu
/tmp/dist-test-taskItYPJR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315935617919-15814-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315935617919-15814-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315935617919-15814-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315935617919-15814-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.15.113.148:47973/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.113.130:43941
--local_ip_for_outbound_sockets=127.15.113.130
--tserver_master_addrs=127.15.113.190:34121
--webserver_port=38683
--webserver_interface=127.15.113.130
--keytab_file=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.130.keytab
--principal=kudu/127.15.113.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.113.148:40529
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.15.113.148:47973/kms with env {KRB5CCNAME=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315935617919-15814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315935617919-15814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315935617919-15814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 22:55:36 dist-test-slave-ncp9 krb5kdc[2126](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316136, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.113.129@KRBTEST.COM for kudu/127.15.113.190@KRBTEST.COM
I20260809 22:55:36.544070 3985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:55:36.502137 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.113.129:40435 (local address 127.15.113.190:34121)
0809 22:55:36.517300 (+ 15163us) server_negotiation.cc:206] Beginning negotiation
0809 22:55:36.517321 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0809 22:55:36.518801 (+ 1480us) server_negotiation.cc:407] Connection header received
0809 22:55:36.521547 (+ 2746us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:55:36.521566 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:55:36.522081 (+ 515us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:55:36.522505 (+ 424us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:55:36.524622 (+ 2117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:55:36.526463 (+ 1841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:55:36.528721 (+ 2258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:55:36.529707 (+ 986us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:55:36.529865 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:55:36.534291 (+ 4426us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:55:36.534301 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:55:36.534319 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:55:36.534364 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:55:36.538173 (+ 3809us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:55:36.539198 (+ 1025us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:55:36.539208 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:55:36.539216 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:55:36.539426 (+ 210us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:55:36.540001 (+ 575us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:55:36.540007 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:55:36.540010 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:55:36.540708 (+ 698us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:55:36.540924 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0809 22:55:36.542216 (+ 1292us) server_negotiation.cc:299] Negotiation successful
0809 22:55:36.542663 (+ 447us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14855,"thread_start_us":15035,"threads_started":1}
I20260809 22:55:36.545136 3986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:55:36.502335 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:34121 (local address 127.15.113.129:40435)
0809 22:55:36.518151 (+ 15816us) negotiation.cc:107] Waiting for socket to connect
0809 22:55:36.518211 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0809 22:55:36.520690 (+ 2479us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:55:36.523218 (+ 2528us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:55:36.523230 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:55:36.523787 (+ 557us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:55:36.524403 (+ 616us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:55:36.524429 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:55:36.526653 (+ 2224us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:55:36.526658 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:55:36.528562 (+ 1904us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:55:36.528577 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:55:36.530266 (+ 1689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:55:36.530276 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:55:36.530575 (+ 299us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:55:36.531408 (+ 833us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:55:36.531437 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:55:36.534019 (+ 2582us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:55:36.538363 (+ 4344us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:55:36.538374 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:55:36.538390 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:55:36.539014 (+ 624us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:55:36.539573 (+ 559us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:55:36.539577 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:55:36.539581 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:55:36.539840 (+ 259us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:55:36.541064 (+ 1224us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:55:36.541074 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:55:36.541535 (+ 461us) client_negotiation.cc:769] Sending connection context
0809 22:55:36.541928 (+ 393us) client_negotiation.cc:240] Negotiation successful
0809 22:55:36.542573 (+ 645us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15308,"thread_start_us":15050,"threads_started":1}
I20260809 22:55:36.549254 3982 heartbeater.cc:347] Connected to a master server at 127.15.113.190:34121
I20260809 22:55:36.549630 3982 heartbeater.cc:464] Registering TS with master...
I20260809 22:55:36.550642 3982 heartbeater.cc:511] Master 127.15.113.190:34121 requested a full tablet report, sending...
W20260809 22:55:36.554361 3982 heartbeater.cc:644] Failed to heartbeat to 127.15.113.190:34121 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260809 22:55:36.554632 3982 heartbeater.cc:464] Registering TS with master...
I20260809 22:55:36.554935 3982 heartbeater.cc:511] Master 127.15.113.190:34121 requested a full tablet report, sending...
I20260809 22:55:36.556381 3982 heartbeater.cc:464] Registering TS with master...
I20260809 22:55:36.556757 3982 heartbeater.cc:511] Master 127.15.113.190:34121 requested a full tablet report, sending...
W20260809 22:55:36.558048 3982 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260809 22:55:36.941156 3987 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:55:36.941512 3987 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:55:36.941720 3987 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:55:36.952849 3987 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:55:36.953061 3987 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:55:36.953260 3987 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.113.130
Aug 09 22:55:36 dist-test-slave-ncp9 krb5kdc[2126](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786316136, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.113.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:55:36.968546 3987 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.113.148:40529
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315935617919-15814-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315935617919-15814-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.15.113.148:47973/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.130.keytab
--principal=kudu/127.15.113.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.113.130:43941
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315935617919-15814-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.113.130
--webserver_port=38683
--tserver_master_addrs=127.15.113.190:34121
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.113.130
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315935617919-15814-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:55:36.970450 3987 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:55:36.972272 3987 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:55:36.993340 4016 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:55:36.994958 4017 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:55:36.998020 4024 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:55:36.999047 3987 server_base.cc:1034] running on GCE node
I20260809 22:55:37.000015 3987 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:55:37.001344 3987 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:55:37.002671 3987 hybrid_clock.cc:648] HybridClock initialized: now 1786316137002571 us; error 85 us; skew 500 ppm
I20260809 22:55:37.003093 3987 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:55:37 dist-test-slave-ncp9 krb5kdc[2126](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316137, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.113.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:55:37.007277 3987 init.cc:377] Logged in from keytab as kudu/127.15.113.130@KRBTEST.COM (short username kudu)
I20260809 22:55:37.013278 3987 webserver.cc:460] Webserver started at http://127.15.113.130:38683/ using document root <none> and password file <none>
I20260809 22:55:37.014216 3987 fs_manager.cc:359] Metadata directory not provided
I20260809 22:55:37.014375 3987 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 09 22:55:37 dist-test-slave-ncp9 krb5kdc[2126](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316137, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.113.130@KRBTEST.COM for HTTP/127.15.113.148@KRBTEST.COM
I20260809 22:55:37.056713 3987 fs_manager.cc:709] Time spent opening directory manager: real 0.012s user 0.003s sys 0.008s
I20260809 22:55:37.060730 4038 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:55:37.062567 3987 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260809 22:55:37.062747 3987 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315935617919-15814-0/minicluster-data/ts-1/data,/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315935617919-15814-0/minicluster-data/ts-1/wal
uuid: "a619374402094f629bb96421437ea9aa"
format_stamp: "Formatted at 2026-08-09 22:55:33 on dist-test-slave-ncp9"
server_key: "62a69f8afb08edb69dfa35b8043cff73"
server_key_iv: "dc7899767d9d16b361ccf8670c43b0d1"
server_key_version: "kuduclusterkey@0"
I20260809 22:55:37.064311 3987 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315935617919-15814-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315935617919-15814-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315935617919-15814-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:55:37.092428 3987 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:55:37.098049 3987 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:55:37.098362 3987 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:55:37.099661 3987 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:55:37.101477 3987 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:55:37.101584 3987 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:55:37.101691 3987 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:55:37.101768 3987 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:55:37.175580 3987 rpc_server.cc:307] RPC server started. Bound to: 127.15.113.130:43941
I20260809 22:55:37.175639 4211 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.113.130:43941 every 8 connection(s)
I20260809 22:55:37.177605 3987 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315935617919-15814-0/minicluster-data/ts-1/data/info.pb
I20260809 22:55:37.182384 15814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskItYPJR/build/asan/bin/kudu as pid 3987
I20260809 22:55:37.183228 15814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskItYPJR/build/asan/bin/kudu
/tmp/dist-test-taskItYPJR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315935617919-15814-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315935617919-15814-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315935617919-15814-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315935617919-15814-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.15.113.148:47973/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.113.131:36419
--local_ip_for_outbound_sockets=127.15.113.131
--tserver_master_addrs=127.15.113.190:34121
--webserver_port=40959
--webserver_interface=127.15.113.131
--keytab_file=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.131.keytab
--principal=kudu/127.15.113.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.113.148:40529
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.15.113.148:47973/kms with env {KRB5CCNAME=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315935617919-15814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315935617919-15814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315935617919-15814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 22:55:37 dist-test-slave-ncp9 krb5kdc[2126](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316137, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.113.130@KRBTEST.COM for kudu/127.15.113.190@KRBTEST.COM
I20260809 22:55:37.219704 4216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:55:37.181394 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:34121 (local address 127.15.113.130:46541)
0809 22:55:37.187400 (+ 6006us) negotiation.cc:107] Waiting for socket to connect
0809 22:55:37.187450 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0809 22:55:37.189213 (+ 1763us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:55:37.198143 (+ 8930us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:55:37.198157 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:55:37.198840 (+ 683us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:55:37.199721 (+ 881us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:55:37.199751 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:55:37.201365 (+ 1614us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:55:37.201371 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:55:37.203454 (+ 2083us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:55:37.203495 (+ 41us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:55:37.204773 (+ 1278us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:55:37.204781 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:55:37.205094 (+ 313us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:55:37.205902 (+ 808us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:55:37.205934 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:55:37.207996 (+ 2062us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:55:37.214638 (+ 6642us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:55:37.214647 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:55:37.214664 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:55:37.215268 (+ 604us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:55:37.215734 (+ 466us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:55:37.215739 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:55:37.215743 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:55:37.216012 (+ 269us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:55:37.216817 (+ 805us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:55:37.216825 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:55:37.217274 (+ 449us) client_negotiation.cc:769] Sending connection context
0809 22:55:37.217619 (+ 345us) client_negotiation.cc:240] Negotiation successful
0809 22:55:37.218193 (+ 574us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5697,"spinlock_wait_cycles":154368,"thread_start_us":11399,"threads_started":1}
I20260809 22:55:37.220781 4215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:55:37.180571 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.113.130:46541 (local address 127.15.113.190:34121)
0809 22:55:37.197303 (+ 16732us) server_negotiation.cc:206] Beginning negotiation
0809 22:55:37.197309 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:55:37.197335 (+ 26us) server_negotiation.cc:407] Connection header received
0809 22:55:37.197409 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:55:37.197414 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:55:37.197519 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:55:37.197647 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:55:37.199955 (+ 2308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:55:37.201159 (+ 1204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:55:37.203660 (+ 2501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:55:37.204293 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:55:37.204426 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:55:37.208203 (+ 3777us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:55:37.208210 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:55:37.208215 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:55:37.208259 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:55:37.214476 (+ 6217us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:55:37.215419 (+ 943us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:55:37.215425 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:55:37.215429 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:55:37.215593 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:55:37.216155 (+ 562us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:55:37.216160 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:55:37.216164 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:55:37.216576 (+ 412us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:55:37.216718 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0809 22:55:37.220188 (+ 3470us) server_negotiation.cc:299] Negotiation successful
0809 22:55:37.220418 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16589,"thread_start_us":16448,"threads_started":1}
I20260809 22:55:37.223104 4212 heartbeater.cc:347] Connected to a master server at 127.15.113.190:34121
I20260809 22:55:37.223428 4212 heartbeater.cc:464] Registering TS with master...
I20260809 22:55:37.224334 4212 heartbeater.cc:511] Master 127.15.113.190:34121 requested a full tablet report, sending...
I20260809 22:55:37.226665 3690 ts_manager.cc:194] Registered new tserver with Master: a619374402094f629bb96421437ea9aa (127.15.113.130:43941)
I20260809 22:55:37.229089 3690 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.113.130@KRBTEST.COM'} at 127.15.113.130:46541
Aug 09 22:55:37 dist-test-slave-ncp9 krb5kdc[2126](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316136, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.113.190@KRBTEST.COM for HTTP/127.15.113.148@KRBTEST.COM
I20260809 22:55:37.558209 3982 heartbeater.cc:464] Registering TS with master...
I20260809 22:55:37.558722 3982 heartbeater.cc:511] Master 127.15.113.190:34121 requested a full tablet report, sending...
I20260809 22:55:37.565824 3690 ts_manager.cc:194] Registered new tserver with Master: 1e145cb790454107aeda329005bdc491 (127.15.113.129:34291)
I20260809 22:55:37.567051 3690 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.113.129@KRBTEST.COM'} at 127.15.113.129:40435
W20260809 22:55:37.656647 4217 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:55:37.657202 4217 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:55:37.657431 4217 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:55:37.669077 4217 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:55:37.669282 4217 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:55:37.669513 4217 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.113.131
I20260809 22:55:37.684728 4217 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.113.148:40529
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315935617919-15814-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315935617919-15814-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.15.113.148:47973/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.131.keytab
--principal=kudu/127.15.113.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.113.131:36419
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315935617919-15814-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.113.131
--webserver_port=40959
--tserver_master_addrs=127.15.113.190:34121
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.113.131
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315935617919-15814-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:55:37.686590 4217 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:55:37.688376 4217 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:55:37.706980 4225 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:55:37.711974 4226 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:55:37.713567 4228 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:55:37.714450 4217 server_base.cc:1034] running on GCE node
I20260809 22:55:37.715263 4217 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:55:37.716934 4217 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:55:37.718420 4217 hybrid_clock.cc:648] HybridClock initialized: now 1786316137718343 us; error 58 us; skew 500 ppm
I20260809 22:55:37.718806 4217 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:55:37 dist-test-slave-ncp9 krb5kdc[2126](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316137, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.113.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:55:37.723110 4217 init.cc:377] Logged in from keytab as kudu/127.15.113.131@KRBTEST.COM (short username kudu)
I20260809 22:55:37.726365 4217 webserver.cc:460] Webserver started at http://127.15.113.131:40959/ using document root <none> and password file <none>
I20260809 22:55:37.727303 4217 fs_manager.cc:359] Metadata directory not provided
I20260809 22:55:37.727456 4217 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 09 22:55:37 dist-test-slave-ncp9 krb5kdc[2126](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316137, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.113.131@KRBTEST.COM for HTTP/127.15.113.148@KRBTEST.COM
I20260809 22:55:37.757783 4217 fs_manager.cc:709] Time spent opening directory manager: real 0.008s user 0.005s sys 0.000s
I20260809 22:55:37.762427 4239 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:55:37.764261 4217 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.001s sys 0.003s
I20260809 22:55:37.764560 4217 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315935617919-15814-0/minicluster-data/ts-2/data,/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315935617919-15814-0/minicluster-data/ts-2/wal
uuid: "6e9036e451a84c2aa357fa45343e4c7c"
format_stamp: "Formatted at 2026-08-09 22:55:34 on dist-test-slave-ncp9"
server_key: "0e5fef32080dd1eda8374921de1513f8"
server_key_iv: "5a49346f815a5959a264bc9ce2d79960"
server_key_version: "kuduclusterkey@0"
I20260809 22:55:37.766047 4217 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315935617919-15814-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315935617919-15814-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315935617919-15814-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
Aug 09 22:55:37 dist-test-slave-ncp9 krb5kdc[2126](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316136, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.113.190@KRBTEST.COM for HTTP/127.15.113.148@KRBTEST.COM
I20260809 22:55:37.800135 4217 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:55:37.805702 4217 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:55:37.806093 4217 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:55:37.809242 4217 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:55:37.811506 4217 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:55:37.811646 4217 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:55:37.811775 4217 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:55:37.811877 4217 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:55:38.146010 4217 rpc_server.cc:307] RPC server started. Bound to: 127.15.113.131:36419
I20260809 22:55:38.146067 4430 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.113.131:36419 every 8 connection(s)
I20260809 22:55:38.148113 4217 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315935617919-15814-0/minicluster-data/ts-2/data/info.pb
I20260809 22:55:38.156934 15814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskItYPJR/build/asan/bin/kudu as pid 4217
Aug 09 22:55:38 dist-test-slave-ncp9 krb5kdc[2126](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316137, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.113.131@KRBTEST.COM for kudu/127.15.113.190@KRBTEST.COM
I20260809 22:55:38.185678 4436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:55:38.152232 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.113.131:50695 (local address 127.15.113.190:34121)
0809 22:55:38.165284 (+ 13052us) server_negotiation.cc:206] Beginning negotiation
0809 22:55:38.165290 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:55:38.165311 (+ 21us) server_negotiation.cc:407] Connection header received
0809 22:55:38.165378 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:55:38.165383 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:55:38.165503 (+ 120us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:55:38.165636 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:55:38.167753 (+ 2117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:55:38.168996 (+ 1243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:55:38.171440 (+ 2444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:55:38.172107 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:55:38.173219 (+ 1112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:55:38.176534 (+ 3315us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:55:38.176540 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:55:38.176544 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:55:38.176576 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:55:38.181987 (+ 5411us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:55:38.182862 (+ 875us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:55:38.182866 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:55:38.182869 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:55:38.183020 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:55:38.183634 (+ 614us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:55:38.183640 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:55:38.183643 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:55:38.184041 (+ 398us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:55:38.184194 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0809 22:55:38.185037 (+ 843us) server_negotiation.cc:299] Negotiation successful
0809 22:55:38.185282 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12942,"thread_start_us":12821,"threads_started":1}
I20260809 22:55:38.186631 4434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:55:38.151588 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:34121 (local address 127.15.113.131:50695)
0809 22:55:38.162233 (+ 10645us) negotiation.cc:107] Waiting for socket to connect
0809 22:55:38.162296 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0809 22:55:38.164731 (+ 2435us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:55:38.165952 (+ 1221us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:55:38.165964 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:55:38.166613 (+ 649us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:55:38.167530 (+ 917us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:55:38.167559 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:55:38.169177 (+ 1618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:55:38.169183 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:55:38.171211 (+ 2028us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:55:38.171227 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:55:38.172377 (+ 1150us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:55:38.172383 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:55:38.172683 (+ 300us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:55:38.173517 (+ 834us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:55:38.173544 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:55:38.175995 (+ 2451us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:55:38.182133 (+ 6138us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:55:38.182142 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:55:38.182157 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:55:38.182708 (+ 551us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:55:38.183183 (+ 475us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:55:38.183188 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:55:38.183192 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:55:38.183466 (+ 274us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:55:38.184185 (+ 719us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:55:38.184193 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:55:38.184595 (+ 402us) client_negotiation.cc:769] Sending connection context
0809 22:55:38.184884 (+ 289us) client_negotiation.cc:240] Negotiation successful
0809 22:55:38.185402 (+ 518us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10314,"thread_start_us":10100,"threads_started":1}
I20260809 22:55:38.188665 4432 heartbeater.cc:347] Connected to a master server at 127.15.113.190:34121
I20260809 22:55:38.188988 4432 heartbeater.cc:464] Registering TS with master...
I20260809 22:55:38.189831 4432 heartbeater.cc:511] Master 127.15.113.190:34121 requested a full tablet report, sending...
I20260809 22:55:38.191452 3690 ts_manager.cc:194] Registered new tserver with Master: 6e9036e451a84c2aa357fa45343e4c7c (127.15.113.131:36419)
I20260809 22:55:38.192026 4439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:55:38.165256 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58590 (local address 127.15.113.190:34121)
0809 22:55:38.167942 (+ 2686us) server_negotiation.cc:206] Beginning negotiation
0809 22:55:38.167948 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:55:38.167967 (+ 19us) server_negotiation.cc:407] Connection header received
0809 22:55:38.168025 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:55:38.168030 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:55:38.168137 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:55:38.168240 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:55:38.170648 (+ 2408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:55:38.171837 (+ 1189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:55:38.174155 (+ 2318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:55:38.176223 (+ 2068us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:55:38.176316 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:55:38.182394 (+ 6078us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:55:38.182401 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:55:38.182406 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:55:38.182448 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:55:38.187680 (+ 5232us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:55:38.188593 (+ 913us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:55:38.188599 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:55:38.188602 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:55:38.188751 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:55:38.189249 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:55:38.189254 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:55:38.189258 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:55:38.189591 (+ 333us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:55:38.189854 (+ 263us) server_negotiation.cc:1035] Waiting for connection context
0809 22:55:38.191113 (+ 1259us) server_negotiation.cc:299] Negotiation successful
0809 22:55:38.191339 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2564,"thread_start_us":287,"threads_started":1}
I20260809 22:55:38.192487 3690 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.113.131@KRBTEST.COM'} at 127.15.113.131:50695
I20260809 22:55:38.195473 15814 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 22:55:38.214619 4439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:55:38.199009 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58592 (local address 127.15.113.190:34121)
0809 22:55:38.199180 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0809 22:55:38.199185 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:55:38.199696 (+ 511us) server_negotiation.cc:407] Connection header received
0809 22:55:38.199843 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:55:38.199848 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:55:38.199914 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:55:38.200003 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:55:38.201957 (+ 1954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:55:38.202925 (+ 968us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:55:38.205006 (+ 2081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:55:38.205578 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:55:38.205682 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:55:38.208867 (+ 3185us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:55:38.208873 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:55:38.208877 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:55:38.208932 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:55:38.211899 (+ 2967us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:55:38.212631 (+ 732us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:55:38.212636 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:55:38.212639 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:55:38.212776 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:55:38.213167 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:55:38.213171 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:55:38.213173 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:55:38.213465 (+ 292us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:55:38.213690 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0809 22:55:38.213974 (+ 284us) server_negotiation.cc:299] Negotiation successful
0809 22:55:38.214170 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260809 22:55:38.219964 3689 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58592:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 22:55:38.229660 3689 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 22:55:38.234100 4212 heartbeater.cc:503] Master 127.15.113.190:34121 was elected leader, sending a full tablet report...
I20260809 22:55:38.282063 4456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:55:38.253274 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.131:36419 (local address 127.0.0.1:43576)
0809 22:55:38.265571 (+ 12297us) negotiation.cc:107] Waiting for socket to connect
0809 22:55:38.265625 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0809 22:55:38.265853 (+ 228us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:55:38.267178 (+ 1325us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:55:38.267187 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:55:38.267224 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:55:38.267696 (+ 472us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:55:38.267710 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:55:38.272466 (+ 4756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:55:38.272473 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:55:38.275045 (+ 2572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:55:38.275064 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:55:38.281109 (+ 6045us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:55:38.281117 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:55:38.281323 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:55:38.281385 (+ 62us) client_negotiation.cc:769] Sending connection context
0809 22:55:38.281506 (+ 121us) client_negotiation.cc:240] Negotiation successful
0809 22:55:38.281701 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11976,"thread_start_us":11810,"threads_started":1}
I20260809 22:55:38.282526 4457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:55:38.253553 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43576 (local address 127.15.113.131:36419)
0809 22:55:38.266431 (+ 12878us) server_negotiation.cc:206] Beginning negotiation
0809 22:55:38.266441 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 22:55:38.266476 (+ 35us) server_negotiation.cc:407] Connection header received
0809 22:55:38.266589 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:55:38.266598 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:55:38.266891 (+ 293us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:55:38.267921 (+ 1030us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:55:38.268258 (+ 337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:55:38.271677 (+ 3419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:55:38.275302 (+ 3625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:55:38.276863 (+ 1561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:55:38.281741 (+ 4878us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:55:38.281934 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0809 22:55:38.282013 (+ 79us) server_negotiation.cc:299] Negotiation successful
0809 22:55:38.282178 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12694,"thread_start_us":18552,"threads_started":1}
I20260809 22:55:38.286583 4324 tablet_service.cc:1467] Processing CreateTablet for tablet f6a724f816114c40b83363be67b7d5a4 (DEFAULT_TABLE table=test-table [id=c7a57dcf96fa44fda5800def39e7d43a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:55:38.288178 4324 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f6a724f816114c40b83363be67b7d5a4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:55:38.292585 4456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:55:38.265984 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.129:34291 (local address 127.0.0.1:54410)
0809 22:55:38.283025 (+ 17041us) negotiation.cc:107] Waiting for socket to connect
0809 22:55:38.283044 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 22:55:38.283151 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:55:38.284094 (+ 943us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:55:38.284098 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:55:38.284125 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:55:38.284272 (+ 147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:55:38.284280 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:55:38.286920 (+ 2640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:55:38.286925 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:55:38.288735 (+ 1810us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:55:38.288752 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:55:38.290841 (+ 2089us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:55:38.290847 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:55:38.292034 (+ 1187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:55:38.292070 (+ 36us) client_negotiation.cc:769] Sending connection context
0809 22:55:38.292155 (+ 85us) client_negotiation.cc:240] Negotiation successful
0809 22:55:38.292292 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16940,"thread_start_us":5913,"threads_started":1}
I20260809 22:55:38.293860 4463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:55:38.266324 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54410 (local address 127.15.113.129:34291)
0809 22:55:38.281560 (+ 15236us) server_negotiation.cc:206] Beginning negotiation
0809 22:55:38.281572 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 22:55:38.283268 (+ 1696us) server_negotiation.cc:407] Connection header received
0809 22:55:38.283380 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:55:38.283388 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:55:38.283665 (+ 277us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:55:38.283851 (+ 186us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:55:38.284447 (+ 596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:55:38.286772 (+ 2325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:55:38.288969 (+ 2197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:55:38.290385 (+ 1416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:55:38.290582 (+ 197us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:55:38.290742 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0809 22:55:38.293351 (+ 2609us) server_negotiation.cc:299] Negotiation successful
0809 22:55:38.293518 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13784,"thread_start_us":15000,"threads_started":1}
I20260809 22:55:38.297859 3888 tablet_service.cc:1467] Processing CreateTablet for tablet f6a724f816114c40b83363be67b7d5a4 (DEFAULT_TABLE table=test-table [id=c7a57dcf96fa44fda5800def39e7d43a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:55:38.300477 3888 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f6a724f816114c40b83363be67b7d5a4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:55:38.312223 4464 tablet_bootstrap.cc:492] T f6a724f816114c40b83363be67b7d5a4 P 6e9036e451a84c2aa357fa45343e4c7c: Bootstrap starting.
I20260809 22:55:38.315588 4462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:55:38.254507 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.130:43941 (local address 127.0.0.1:36644)
0809 22:55:38.272144 (+ 17637us) negotiation.cc:107] Waiting for socket to connect
0809 22:55:38.272164 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 22:55:38.272279 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:55:38.297388 (+ 25109us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:55:38.297394 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:55:38.297421 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:55:38.297788 (+ 367us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:55:38.297800 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:55:38.310960 (+ 13160us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:55:38.310978 (+ 18us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:55:38.312986 (+ 2008us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:55:38.313001 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:55:38.314828 (+ 1827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:55:38.314835 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:55:38.315018 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:55:38.315057 (+ 39us) client_negotiation.cc:769] Sending connection context
0809 22:55:38.315128 (+ 71us) client_negotiation.cc:240] Negotiation successful
0809 22:55:38.315266 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17519,"thread_start_us":49352,"threads_started":1}
I20260809 22:55:38.316926 4459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:55:38.255055 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36644 (local address 127.15.113.130:43941)
0809 22:55:38.296499 (+ 41444us) server_negotiation.cc:206] Beginning negotiation
0809 22:55:38.296507 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 22:55:38.296535 (+ 28us) server_negotiation.cc:407] Connection header received
0809 22:55:38.296623 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:55:38.296630 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:55:38.297209 (+ 579us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:55:38.297643 (+ 434us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:55:38.309042 (+ 11399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:55:38.310777 (+ 1735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:55:38.313207 (+ 2430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:55:38.314577 (+ 1370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:55:38.316203 (+ 1626us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:55:38.316368 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0809 22:55:38.316434 (+ 66us) server_negotiation.cc:299] Negotiation successful
0809 22:55:38.316587 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41275,"thread_start_us":40899,"threads_started":1}
I20260809 22:55:38.317037 4464 tablet_bootstrap.cc:654] T f6a724f816114c40b83363be67b7d5a4 P 6e9036e451a84c2aa357fa45343e4c7c: Neither blocks nor log segments found. Creating new log.
I20260809 22:55:38.320628 4107 tablet_service.cc:1467] Processing CreateTablet for tablet f6a724f816114c40b83363be67b7d5a4 (DEFAULT_TABLE table=test-table [id=c7a57dcf96fa44fda5800def39e7d43a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:55:38.322096 4107 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f6a724f816114c40b83363be67b7d5a4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:55:38.330068 4464 log.cc:824] T f6a724f816114c40b83363be67b7d5a4 P 6e9036e451a84c2aa357fa45343e4c7c: Log is configured to *not* fsync() on all Append() calls
I20260809 22:55:38.340794 4464 tablet_bootstrap.cc:492] T f6a724f816114c40b83363be67b7d5a4 P 6e9036e451a84c2aa357fa45343e4c7c: No bootstrap required, opened a new log
I20260809 22:55:38.341264 4464 ts_tablet_manager.cc:1397] T f6a724f816114c40b83363be67b7d5a4 P 6e9036e451a84c2aa357fa45343e4c7c: Time spent bootstrapping tablet: real 0.029s user 0.009s sys 0.000s
I20260809 22:55:38.349565 4467 tablet_bootstrap.cc:492] T f6a724f816114c40b83363be67b7d5a4 P a619374402094f629bb96421437ea9aa: Bootstrap starting.
I20260809 22:55:38.354018 4467 tablet_bootstrap.cc:654] T f6a724f816114c40b83363be67b7d5a4 P a619374402094f629bb96421437ea9aa: Neither blocks nor log segments found. Creating new log.
I20260809 22:55:38.354848 4465 tablet_bootstrap.cc:492] T f6a724f816114c40b83363be67b7d5a4 P 1e145cb790454107aeda329005bdc491: Bootstrap starting.
I20260809 22:55:38.358016 4465 tablet_bootstrap.cc:654] T f6a724f816114c40b83363be67b7d5a4 P 1e145cb790454107aeda329005bdc491: Neither blocks nor log segments found. Creating new log.
I20260809 22:55:38.358700 4464 raft_consensus.cc:348] T f6a724f816114c40b83363be67b7d5a4 P 6e9036e451a84c2aa357fa45343e4c7c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e9036e451a84c2aa357fa45343e4c7c" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 36419 } } peers { permanent_uuid: "1e145cb790454107aeda329005bdc491" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 34291 } } peers { permanent_uuid: "a619374402094f629bb96421437ea9aa" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 43941 } }
I20260809 22:55:38.359141 4464 raft_consensus.cc:374] T f6a724f816114c40b83363be67b7d5a4 P 6e9036e451a84c2aa357fa45343e4c7c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:55:38.359246 4464 raft_consensus.cc:729] T f6a724f816114c40b83363be67b7d5a4 P 6e9036e451a84c2aa357fa45343e4c7c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e9036e451a84c2aa357fa45343e4c7c, State: Initialized, Role: FOLLOWER
I20260809 22:55:38.359402 4465 log.cc:824] T f6a724f816114c40b83363be67b7d5a4 P 1e145cb790454107aeda329005bdc491: Log is configured to *not* fsync() on all Append() calls
I20260809 22:55:38.359726 4467 log.cc:824] T f6a724f816114c40b83363be67b7d5a4 P a619374402094f629bb96421437ea9aa: Log is configured to *not* fsync() on all Append() calls
I20260809 22:55:38.360625 4464 consensus_queue.cc:260] T f6a724f816114c40b83363be67b7d5a4 P 6e9036e451a84c2aa357fa45343e4c7c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e9036e451a84c2aa357fa45343e4c7c" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 36419 } } peers { permanent_uuid: "1e145cb790454107aeda329005bdc491" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 34291 } } peers { permanent_uuid: "a619374402094f629bb96421437ea9aa" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 43941 } }
I20260809 22:55:38.366737 4467 tablet_bootstrap.cc:492] T f6a724f816114c40b83363be67b7d5a4 P a619374402094f629bb96421437ea9aa: No bootstrap required, opened a new log
I20260809 22:55:38.367045 4467 ts_tablet_manager.cc:1397] T f6a724f816114c40b83363be67b7d5a4 P a619374402094f629bb96421437ea9aa: Time spent bootstrapping tablet: real 0.018s user 0.009s sys 0.000s
I20260809 22:55:38.373798 4432 heartbeater.cc:503] Master 127.15.113.190:34121 was elected leader, sending a full tablet report...
I20260809 22:55:38.373997 4464 ts_tablet_manager.cc:1428] T f6a724f816114c40b83363be67b7d5a4 P 6e9036e451a84c2aa357fa45343e4c7c: Time spent starting tablet: real 0.033s user 0.011s sys 0.008s
I20260809 22:55:38.378906 4467 raft_consensus.cc:348] T f6a724f816114c40b83363be67b7d5a4 P a619374402094f629bb96421437ea9aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e9036e451a84c2aa357fa45343e4c7c" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 36419 } } peers { permanent_uuid: "1e145cb790454107aeda329005bdc491" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 34291 } } peers { permanent_uuid: "a619374402094f629bb96421437ea9aa" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 43941 } }
I20260809 22:55:38.380033 4467 raft_consensus.cc:374] T f6a724f816114c40b83363be67b7d5a4 P a619374402094f629bb96421437ea9aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:55:38.380187 4467 raft_consensus.cc:729] T f6a724f816114c40b83363be67b7d5a4 P a619374402094f629bb96421437ea9aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a619374402094f629bb96421437ea9aa, State: Initialized, Role: FOLLOWER
I20260809 22:55:38.380911 4467 consensus_queue.cc:260] T f6a724f816114c40b83363be67b7d5a4 P a619374402094f629bb96421437ea9aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e9036e451a84c2aa357fa45343e4c7c" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 36419 } } peers { permanent_uuid: "1e145cb790454107aeda329005bdc491" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 34291 } } peers { permanent_uuid: "a619374402094f629bb96421437ea9aa" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 43941 } }
I20260809 22:55:38.382535 4465 tablet_bootstrap.cc:492] T f6a724f816114c40b83363be67b7d5a4 P 1e145cb790454107aeda329005bdc491: No bootstrap required, opened a new log
I20260809 22:55:38.382881 4465 ts_tablet_manager.cc:1397] T f6a724f816114c40b83363be67b7d5a4 P 1e145cb790454107aeda329005bdc491: Time spent bootstrapping tablet: real 0.028s user 0.012s sys 0.000s
I20260809 22:55:38.393211 4465 raft_consensus.cc:348] T f6a724f816114c40b83363be67b7d5a4 P 1e145cb790454107aeda329005bdc491 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e9036e451a84c2aa357fa45343e4c7c" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 36419 } } peers { permanent_uuid: "1e145cb790454107aeda329005bdc491" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 34291 } } peers { permanent_uuid: "a619374402094f629bb96421437ea9aa" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 43941 } }
I20260809 22:55:38.393712 4465 raft_consensus.cc:374] T f6a724f816114c40b83363be67b7d5a4 P 1e145cb790454107aeda329005bdc491 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:55:38.393829 4465 raft_consensus.cc:729] T f6a724f816114c40b83363be67b7d5a4 P 1e145cb790454107aeda329005bdc491 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1e145cb790454107aeda329005bdc491, State: Initialized, Role: FOLLOWER
I20260809 22:55:38.394560 4465 consensus_queue.cc:260] T f6a724f816114c40b83363be67b7d5a4 P 1e145cb790454107aeda329005bdc491 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e9036e451a84c2aa357fa45343e4c7c" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 36419 } } peers { permanent_uuid: "1e145cb790454107aeda329005bdc491" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 34291 } } peers { permanent_uuid: "a619374402094f629bb96421437ea9aa" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 43941 } }
I20260809 22:55:38.396952 4467 ts_tablet_manager.cc:1428] T f6a724f816114c40b83363be67b7d5a4 P a619374402094f629bb96421437ea9aa: Time spent starting tablet: real 0.030s user 0.022s sys 0.004s
I20260809 22:55:38.400877 4465 ts_tablet_manager.cc:1428] T f6a724f816114c40b83363be67b7d5a4 P 1e145cb790454107aeda329005bdc491: Time spent starting tablet: real 0.018s user 0.009s sys 0.005s
I20260809 22:55:38.401273 3982 heartbeater.cc:503] Master 127.15.113.190:34121 was elected leader, sending a full tablet report...
W20260809 22:55:38.405781 4433 tablet.cc:2369] T f6a724f816114c40b83363be67b7d5a4 P 6e9036e451a84c2aa357fa45343e4c7c: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 22:55:38.437268 4213 tablet.cc:2369] T f6a724f816114c40b83363be67b7d5a4 P a619374402094f629bb96421437ea9aa: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 22:55:38.521180 3984 tablet.cc:2369] T f6a724f816114c40b83363be67b7d5a4 P 1e145cb790454107aeda329005bdc491: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 22:55:38.723688 4471 raft_consensus.cc:482] T f6a724f816114c40b83363be67b7d5a4 P a619374402094f629bb96421437ea9aa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 22:55:38.723978 4471 raft_consensus.cc:504] T f6a724f816114c40b83363be67b7d5a4 P a619374402094f629bb96421437ea9aa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e9036e451a84c2aa357fa45343e4c7c" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 36419 } } peers { permanent_uuid: "1e145cb790454107aeda329005bdc491" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 34291 } } peers { permanent_uuid: "a619374402094f629bb96421437ea9aa" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 43941 } }
I20260809 22:55:38.725745 4471 leader_election.cc:288] T f6a724f816114c40b83363be67b7d5a4 P a619374402094f629bb96421437ea9aa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6e9036e451a84c2aa357fa45343e4c7c (127.15.113.131:36419), 1e145cb790454107aeda329005bdc491 (127.15.113.129:34291)
I20260809 22:55:38.733359 4457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:55:38.726112 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.113.130:41593 (local address 127.15.113.131:36419)
0809 22:55:38.726269 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0809 22:55:38.726274 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:55:38.726935 (+ 661us) server_negotiation.cc:407] Connection header received
0809 22:55:38.727021 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:55:38.727026 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:55:38.727106 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:55:38.727191 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:55:38.728159 (+ 968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:55:38.729150 (+ 991us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:55:38.731353 (+ 2203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:55:38.732414 (+ 1061us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:55:38.732565 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:55:38.732634 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0809 22:55:38.733035 (+ 401us) server_negotiation.cc:299] Negotiation successful
0809 22:55:38.733148 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260809 22:55:38.733698 4463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:55:38.727205 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.113.130:60205 (local address 127.15.113.129:34291)
0809 22:55:38.727444 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0809 22:55:38.727450 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:55:38.727940 (+ 490us) server_negotiation.cc:407] Connection header received
0809 22:55:38.728099 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:55:38.728104 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:55:38.728175 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:55:38.728269 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:55:38.728786 (+ 517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:55:38.729826 (+ 1040us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:55:38.731890 (+ 2064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:55:38.732744 (+ 854us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:55:38.732915 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:55:38.732988 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0809 22:55:38.733329 (+ 341us) server_negotiation.cc:299] Negotiation successful
0809 22:55:38.733459 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260809 22:55:38.733740 4475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:55:38.726839 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.129:34291 (local address 127.15.113.130:60205)
0809 22:55:38.727813 (+ 974us) negotiation.cc:107] Waiting for socket to connect
0809 22:55:38.727839 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0809 22:55:38.727975 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:55:38.728295 (+ 320us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:55:38.728300 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:55:38.728327 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:55:38.728638 (+ 311us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:55:38.728650 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:55:38.729972 (+ 1322us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:55:38.729981 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:55:38.731741 (+ 1760us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:55:38.731752 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:55:38.733009 (+ 1257us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:55:38.733015 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:55:38.733169 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:55:38.733200 (+ 31us) client_negotiation.cc:769] Sending connection context
0809 22:55:38.733288 (+ 88us) client_negotiation.cc:240] Negotiation successful
0809 22:55:38.733398 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":856,"mutex_wait_us":42,"thread_start_us":586,"threads_started":1}
I20260809 22:55:38.734370 4473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:55:38.726126 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.131:36419 (local address 127.15.113.130:41593)
0809 22:55:38.726693 (+ 567us) negotiation.cc:107] Waiting for socket to connect
0809 22:55:38.726709 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 22:55:38.726858 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:55:38.727532 (+ 674us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:55:38.727537 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:55:38.727570 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:55:38.727978 (+ 408us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:55:38.727988 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:55:38.729273 (+ 1285us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:55:38.729277 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:55:38.731178 (+ 1901us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:55:38.731193 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:55:38.732630 (+ 1437us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:55:38.732636 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:55:38.732797 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:55:38.732832 (+ 35us) client_negotiation.cc:769] Sending connection context
0809 22:55:38.733906 (+ 1074us) client_negotiation.cc:240] Negotiation successful
0809 22:55:38.734035 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":456,"mutex_wait_us":19,"thread_start_us":293,"threads_started":1}
I20260809 22:55:38.734781 3919 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f6a724f816114c40b83363be67b7d5a4" candidate_uuid: "a619374402094f629bb96421437ea9aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1e145cb790454107aeda329005bdc491" is_pre_election: true
I20260809 22:55:38.735190 4361 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f6a724f816114c40b83363be67b7d5a4" candidate_uuid: "a619374402094f629bb96421437ea9aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e9036e451a84c2aa357fa45343e4c7c" is_pre_election: true
I20260809 22:55:38.735354 3919 raft_consensus.cc:2454] T f6a724f816114c40b83363be67b7d5a4 P 1e145cb790454107aeda329005bdc491 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a619374402094f629bb96421437ea9aa in term 0.
I20260809 22:55:38.735615 4361 raft_consensus.cc:2454] T f6a724f816114c40b83363be67b7d5a4 P 6e9036e451a84c2aa357fa45343e4c7c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a619374402094f629bb96421437ea9aa in term 0.
I20260809 22:55:38.736056 4039 leader_election.cc:302] T f6a724f816114c40b83363be67b7d5a4 P a619374402094f629bb96421437ea9aa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1e145cb790454107aeda329005bdc491, a619374402094f629bb96421437ea9aa; no voters:
I20260809 22:55:38.736469 4471 raft_consensus.cc:2781] T f6a724f816114c40b83363be67b7d5a4 P a619374402094f629bb96421437ea9aa [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 22:55:38.736624 4471 raft_consensus.cc:482] T f6a724f816114c40b83363be67b7d5a4 P a619374402094f629bb96421437ea9aa [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 22:55:38.736724 4471 raft_consensus.cc:3037] T f6a724f816114c40b83363be67b7d5a4 P a619374402094f629bb96421437ea9aa [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:55:38.739841 4471 raft_consensus.cc:504] T f6a724f816114c40b83363be67b7d5a4 P a619374402094f629bb96421437ea9aa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e9036e451a84c2aa357fa45343e4c7c" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 36419 } } peers { permanent_uuid: "1e145cb790454107aeda329005bdc491" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 34291 } } peers { permanent_uuid: "a619374402094f629bb96421437ea9aa" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 43941 } }
I20260809 22:55:38.740500 4471 leader_election.cc:288] T f6a724f816114c40b83363be67b7d5a4 P a619374402094f629bb96421437ea9aa [CANDIDATE]: Term 1 election: Requested vote from peers 6e9036e451a84c2aa357fa45343e4c7c (127.15.113.131:36419), 1e145cb790454107aeda329005bdc491 (127.15.113.129:34291)
I20260809 22:55:38.741015 4361 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f6a724f816114c40b83363be67b7d5a4" candidate_uuid: "a619374402094f629bb96421437ea9aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e9036e451a84c2aa357fa45343e4c7c"
I20260809 22:55:38.741226 4361 raft_consensus.cc:3037] T f6a724f816114c40b83363be67b7d5a4 P 6e9036e451a84c2aa357fa45343e4c7c [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:55:38.743929 4361 raft_consensus.cc:2454] T f6a724f816114c40b83363be67b7d5a4 P 6e9036e451a84c2aa357fa45343e4c7c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a619374402094f629bb96421437ea9aa in term 1.
I20260809 22:55:38.744390 3919 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f6a724f816114c40b83363be67b7d5a4" candidate_uuid: "a619374402094f629bb96421437ea9aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1e145cb790454107aeda329005bdc491"
I20260809 22:55:38.744596 3919 raft_consensus.cc:3037] T f6a724f816114c40b83363be67b7d5a4 P 1e145cb790454107aeda329005bdc491 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:55:38.745059 4039 leader_election.cc:302] T f6a724f816114c40b83363be67b7d5a4 P a619374402094f629bb96421437ea9aa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e9036e451a84c2aa357fa45343e4c7c, a619374402094f629bb96421437ea9aa; no voters:
I20260809 22:55:38.745383 4471 raft_consensus.cc:2781] T f6a724f816114c40b83363be67b7d5a4 P a619374402094f629bb96421437ea9aa [term 1 FOLLOWER]: Leader election won for term 1
I20260809 22:55:38.747280 3919 raft_consensus.cc:2454] T f6a724f816114c40b83363be67b7d5a4 P 1e145cb790454107aeda329005bdc491 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a619374402094f629bb96421437ea9aa in term 1.
I20260809 22:55:38.748063 4471 raft_consensus.cc:686] T f6a724f816114c40b83363be67b7d5a4 P a619374402094f629bb96421437ea9aa [term 1 LEADER]: Becoming Leader. State: Replica: a619374402094f629bb96421437ea9aa, State: Running, Role: LEADER
I20260809 22:55:38.748648 4471 consensus_queue.cc:237] T f6a724f816114c40b83363be67b7d5a4 P a619374402094f629bb96421437ea9aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e9036e451a84c2aa357fa45343e4c7c" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 36419 } } peers { permanent_uuid: "1e145cb790454107aeda329005bdc491" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 34291 } } peers { permanent_uuid: "a619374402094f629bb96421437ea9aa" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 43941 } }
I20260809 22:55:38.755859 3689 catalog_manager.cc:5495] T f6a724f816114c40b83363be67b7d5a4 P a619374402094f629bb96421437ea9aa reported cstate change: term changed from 0 to 1, leader changed from <none> to a619374402094f629bb96421437ea9aa (127.15.113.130). New cstate: current_term: 1 leader_uuid: "a619374402094f629bb96421437ea9aa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e9036e451a84c2aa357fa45343e4c7c" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 36419 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1e145cb790454107aeda329005bdc491" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 34291 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a619374402094f629bb96421437ea9aa" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 43941 } health_report { overall_health: HEALTHY } } }
I20260809 22:55:38.843823 4488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:55:38.837564 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43034 (local address 127.15.113.130:43941)
0809 22:55:38.838302 (+ 738us) server_negotiation.cc:206] Beginning negotiation
0809 22:55:38.838308 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:55:38.838328 (+ 20us) server_negotiation.cc:407] Connection header received
0809 22:55:38.838432 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:55:38.838436 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:55:38.838538 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:55:38.838700 (+ 162us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 22:55:38.839042 (+ 342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:55:38.840103 (+ 1061us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:55:38.841481 (+ 1378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:55:38.842055 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:55:38.842165 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:55:38.842606 (+ 441us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 22:55:38.843287 (+ 681us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 22:55:38.843419 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0809 22:55:38.843461 (+ 42us) server_negotiation.cc:299] Negotiation successful
0809 22:55:38.843536 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":604,"thread_start_us":479,"threads_started":1}
I20260809 22:55:38.852321 4361 raft_consensus.cc:1261] T f6a724f816114c40b83363be67b7d5a4 P 6e9036e451a84c2aa357fa45343e4c7c [term 1 FOLLOWER]: Refusing update from remote peer a619374402094f629bb96421437ea9aa: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 22:55:38.852447 3919 raft_consensus.cc:1261] T f6a724f816114c40b83363be67b7d5a4 P 1e145cb790454107aeda329005bdc491 [term 1 FOLLOWER]: Refusing update from remote peer a619374402094f629bb96421437ea9aa: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 22:55:38.853335 4476 consensus_queue.cc:1035] T f6a724f816114c40b83363be67b7d5a4 P a619374402094f629bb96421437ea9aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e9036e451a84c2aa357fa45343e4c7c" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 36419 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 22:55:38.853705 4471 consensus_queue.cc:1035] T f6a724f816114c40b83363be67b7d5a4 P a619374402094f629bb96421437ea9aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "1e145cb790454107aeda329005bdc491" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 34291 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 22:55:38.864995 4489 mvcc.cc:205] Tried to move back new op lower bound from 7316750904729513984 to 7316750904319852544. Current Snapshot: MvccSnapshot[applied={T|T < 7316750904729513984}]
I20260809 22:55:38.889097 3689 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58592:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 22:55:38.889387 3689 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58592:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 22:55:38.893242 3689 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b38b4bc8428a47cea24947c1d328666f: Sending DeleteTablet for 3 replicas of tablet f6a724f816114c40b83363be67b7d5a4
I20260809 22:55:38.894353 4324 tablet_service.cc:1514] Processing DeleteTablet for tablet f6a724f816114c40b83363be67b7d5a4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:55:38 UTC) from {username='kudu', principal='kudu/127.15.113.190@KRBTEST.COM'} at 127.0.0.1:43576
I20260809 22:55:38.894733 3888 tablet_service.cc:1514] Processing DeleteTablet for tablet f6a724f816114c40b83363be67b7d5a4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:55:38 UTC) from {username='kudu', principal='kudu/127.15.113.190@KRBTEST.COM'} at 127.0.0.1:54410
I20260809 22:55:38.895408 4107 tablet_service.cc:1514] Processing DeleteTablet for tablet f6a724f816114c40b83363be67b7d5a4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:55:38 UTC) from {username='kudu', principal='kudu/127.15.113.190@KRBTEST.COM'} at 127.0.0.1:36644
I20260809 22:55:38.906009 15814 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskItYPJR/build/asan/bin/kudu pid 3743
I20260809 22:55:38.918853 4505 tablet_replica.cc:330] T f6a724f816114c40b83363be67b7d5a4 P a619374402094f629bb96421437ea9aa: stopping tablet replica
I20260809 22:55:38.919411 4502 tablet_replica.cc:330] T f6a724f816114c40b83363be67b7d5a4 P 1e145cb790454107aeda329005bdc491: stopping tablet replica
I20260809 22:55:38.919966 4505 raft_consensus.cc:2229] T f6a724f816114c40b83363be67b7d5a4 P a619374402094f629bb96421437ea9aa [term 1 LEADER]: Raft consensus shutting down.
I20260809 22:55:38.920472 4502 raft_consensus.cc:2229] T f6a724f816114c40b83363be67b7d5a4 P 1e145cb790454107aeda329005bdc491 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 22:55:38.921293 4505 raft_consensus.cc:2258] T f6a724f816114c40b83363be67b7d5a4 P a619374402094f629bb96421437ea9aa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 22:55:38.921859 4502 raft_consensus.cc:2258] T f6a724f816114c40b83363be67b7d5a4 P 1e145cb790454107aeda329005bdc491 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 22:55:38.924026 4505 ts_tablet_manager.cc:1905] T f6a724f816114c40b83363be67b7d5a4 P a619374402094f629bb96421437ea9aa: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 22:55:38.924042 4502 ts_tablet_manager.cc:1905] T f6a724f816114c40b83363be67b7d5a4 P 1e145cb790454107aeda329005bdc491: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 22:55:38.925398 4501 tablet_replica.cc:330] T f6a724f816114c40b83363be67b7d5a4 P 6e9036e451a84c2aa357fa45343e4c7c: stopping tablet replica
I20260809 22:55:38.930467 4505 ts_tablet_manager.cc:1918] T f6a724f816114c40b83363be67b7d5a4 P a619374402094f629bb96421437ea9aa: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 22:55:38.930680 4505 log.cc:1196] T f6a724f816114c40b83363be67b7d5a4 P a619374402094f629bb96421437ea9aa: Deleting WAL directory at /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315935617919-15814-0/minicluster-data/ts-1/wal/wals/f6a724f816114c40b83363be67b7d5a4
I20260809 22:55:38.931279 4505 ts_tablet_manager.cc:1939] T f6a724f816114c40b83363be67b7d5a4 P a619374402094f629bb96421437ea9aa: Deleting consensus metadata
I20260809 22:55:38.931413 4502 ts_tablet_manager.cc:1918] T f6a724f816114c40b83363be67b7d5a4 P 1e145cb790454107aeda329005bdc491: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 22:55:38.931953 4501 raft_consensus.cc:2229] T f6a724f816114c40b83363be67b7d5a4 P 6e9036e451a84c2aa357fa45343e4c7c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 22:55:38.932508 4501 raft_consensus.cc:2258] T f6a724f816114c40b83363be67b7d5a4 P 6e9036e451a84c2aa357fa45343e4c7c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 22:55:38.933183 3667 catalog_manager.cc:4841] TS a619374402094f629bb96421437ea9aa (127.15.113.130:43941): tablet f6a724f816114c40b83363be67b7d5a4 (table test-table [id=c7a57dcf96fa44fda5800def39e7d43a]) successfully deleted
I20260809 22:55:38.934312 4501 ts_tablet_manager.cc:1905] T f6a724f816114c40b83363be67b7d5a4 P 6e9036e451a84c2aa357fa45343e4c7c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 22:55:38.942953 4502 log.cc:1196] T f6a724f816114c40b83363be67b7d5a4 P 1e145cb790454107aeda329005bdc491: Deleting WAL directory at /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315935617919-15814-0/minicluster-data/ts-0/wal/wals/f6a724f816114c40b83363be67b7d5a4
I20260809 22:55:38.943538 4502 ts_tablet_manager.cc:1939] T f6a724f816114c40b83363be67b7d5a4 P 1e145cb790454107aeda329005bdc491: Deleting consensus metadata
I20260809 22:55:38.943557 4501 ts_tablet_manager.cc:1918] T f6a724f816114c40b83363be67b7d5a4 P 6e9036e451a84c2aa357fa45343e4c7c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 22:55:38.943928 4501 log.cc:1196] T f6a724f816114c40b83363be67b7d5a4 P 6e9036e451a84c2aa357fa45343e4c7c: Deleting WAL directory at /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315935617919-15814-0/minicluster-data/ts-2/wal/wals/f6a724f816114c40b83363be67b7d5a4
I20260809 22:55:38.944720 4501 ts_tablet_manager.cc:1939] T f6a724f816114c40b83363be67b7d5a4 P 6e9036e451a84c2aa357fa45343e4c7c: Deleting consensus metadata
I20260809 22:55:38.945288 3665 catalog_manager.cc:4841] TS 1e145cb790454107aeda329005bdc491 (127.15.113.129:34291): tablet f6a724f816114c40b83363be67b7d5a4 (table test-table [id=c7a57dcf96fa44fda5800def39e7d43a]) successfully deleted
I20260809 22:55:38.946743 3665 catalog_manager.cc:4841] TS 6e9036e451a84c2aa357fa45343e4c7c (127.15.113.131:36419): tablet f6a724f816114c40b83363be67b7d5a4 (table test-table [id=c7a57dcf96fa44fda5800def39e7d43a]) successfully deleted
I20260809 22:55:38.955957 4463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:55:38.908614 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38944 (local address 127.15.113.129:34291)
0809 22:55:38.908795 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0809 22:55:38.908802 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:55:38.925912 (+ 17110us) server_negotiation.cc:407] Connection header received
0809 22:55:38.926340 (+ 428us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:55:38.926345 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:55:38.926428 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:55:38.926511 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:55:38.933287 (+ 6776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:55:38.942771 (+ 9484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:55:38.945534 (+ 2763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:55:38.946092 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:55:38.946209 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:55:38.949295 (+ 3086us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:55:38.949309 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:55:38.949328 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:55:38.949371 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:55:38.952957 (+ 3586us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:55:38.953747 (+ 790us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:55:38.953756 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:55:38.953760 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:55:38.953921 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:55:38.954334 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:55:38.954340 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:55:38.954343 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:55:38.954788 (+ 445us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:55:38.955097 (+ 309us) server_negotiation.cc:1035] Waiting for connection context
0809 22:55:38.955363 (+ 266us) server_negotiation.cc:299] Negotiation successful
0809 22:55:38.955583 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260809 22:55:38.956609 3976 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:55:39.105604 15814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskItYPJR/build/asan/bin/kudu with pid 3743
I20260809 22:55:39.135062 15814 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskItYPJR/build/asan/bin/kudu pid 3987
I20260809 22:55:39.157615 4488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:55:39.136164 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43040 (local address 127.15.113.130:43941)
0809 22:55:39.136341 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0809 22:55:39.136347 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:55:39.136366 (+ 19us) server_negotiation.cc:407] Connection header received
0809 22:55:39.136432 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:55:39.136437 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:55:39.136518 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:55:39.136626 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:55:39.138394 (+ 1768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:55:39.139644 (+ 1250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:55:39.141844 (+ 2200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:55:39.142552 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:55:39.142676 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:55:39.149314 (+ 6638us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:55:39.149334 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:55:39.149353 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:55:39.149399 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:55:39.153410 (+ 4011us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:55:39.154307 (+ 897us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:55:39.154316 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:55:39.154320 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:55:39.154500 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:55:39.154979 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:55:39.154984 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:55:39.154988 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:55:39.155436 (+ 448us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:55:39.155746 (+ 310us) server_negotiation.cc:1035] Waiting for connection context
0809 22:55:39.156975 (+ 1229us) server_negotiation.cc:299] Negotiation successful
0809 22:55:39.157230 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260809 22:55:39.158713 4207 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:55:39.272083 15814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskItYPJR/build/asan/bin/kudu with pid 3987
I20260809 22:55:39.294848 15814 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskItYPJR/build/asan/bin/kudu pid 4217
I20260809 22:55:39.308568 4511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:55:39.295636 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41556 (local address 127.15.113.131:36419)
0809 22:55:39.296156 (+ 520us) server_negotiation.cc:206] Beginning negotiation
0809 22:55:39.296160 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 22:55:39.296175 (+ 15us) server_negotiation.cc:407] Connection header received
0809 22:55:39.296232 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:55:39.296236 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:55:39.296305 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:55:39.296401 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:55:39.297848 (+ 1447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:55:39.298634 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:55:39.300069 (+ 1435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:55:39.300498 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:55:39.300608 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:55:39.303123 (+ 2515us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:55:39.303136 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:55:39.303153 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:55:39.303188 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:55:39.305847 (+ 2659us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:55:39.306543 (+ 696us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:55:39.306551 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:55:39.306553 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:55:39.306694 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:55:39.307064 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:55:39.307067 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:55:39.307071 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:55:39.307438 (+ 367us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:55:39.307730 (+ 292us) server_negotiation.cc:1035] Waiting for connection context
0809 22:55:39.308030 (+ 300us) server_negotiation.cc:299] Negotiation successful
0809 22:55:39.308233 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":404,"thread_start_us":269,"threads_started":1}
I20260809 22:55:39.309331 4424 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:55:39.395874 15814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskItYPJR/build/asan/bin/kudu with pid 4217
I20260809 22:55:39.417961 15814 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskItYPJR/build/asan/bin/kudu pid 3635
I20260809 22:55:39.419178 3730 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:55:39.530493 15814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskItYPJR/build/asan/bin/kudu with pid 3635
I20260809 22:55:39.560386 15814 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260809 22:55:39.893982 15814 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260809 22:55:39.894197 15814 mini_ranger.cc:66] Stopping Ranger...
I20260809 22:55:40.238103 15814 mini_ranger.cc:68] Stopped Ranger
2026-08-09 22:55:40.238 UTC [2170] LOG: received smart shutdown request
2026-08-09 22:55:40.241 UTC [2170] LOG: background worker "logical replication launcher" (PID 2192) exited with exit code 1
2026-08-09 22:55:40.241 UTC [2179] LOG: shutting down
2026-08-09 22:55:40.373 UTC [2170] LOG: database system is shut down
2026-08-09T22:55:40Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (51957 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786315935617919-15814-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 22:55:40 dist-test-slave-ncp9 krb5kdc[4530](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 22:55:40 dist-test-slave-ncp9 krb5kdc[4530](info): set up 2 sockets
Aug 09 22:55:40 dist-test-slave-ncp9 krb5kdc[4530](info): commencing operation
krb5kdc: starting...
W20260809 22:55:41.706099 15814 mini_kdc.cc:121] Time spent starting KDC: real 1.183s user 0.000s sys 0.034s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786315935617919-15814-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786315935617919-15814-0/krb5kdc/test-user.keytab.
Aug 09 22:55:41 dist-test-slave-ncp9 krb5kdc[4530](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316141, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T22:55:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T22:55:41Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.113.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.113.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.113.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.190.keytab.
Entry for principal kudu/127.15.113.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.190.keytab.
I20260809 22:55:41.948040 15814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskItYPJR/build/asan/bin/kudu
/tmp/dist-test-taskItYPJR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786315935617919-15814-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786315935617919-15814-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786315935617919-15814-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786315935617919-15814-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.113.190:40961
--webserver_interface=127.15.113.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.190.keytab
--principal=kudu/127.15.113.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.113.148:44461
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.113.190:40961,127.15.113.189:38369,127.15.113.188:39007
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786315935617919-15814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786315935617919-15814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786315935617919-15814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:55:42.271970 4548 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:55:42.272248 4548 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:55:42.272378 4548 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:55:42.279666 4548 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 22:55:42.279770 4548 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:55:42.279847 4548 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:55:42.279927 4548 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 22:55:42.279987 4548 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 22:55:42.289525 4548 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.113.148:44461
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786315935617919-15814-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786315935617919-15814-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.113.190:40961,127.15.113.189:38369,127.15.113.188:39007
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.190.keytab
--principal=kudu/127.15.113.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.113.190:40961
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786315935617919-15814-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.113.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786315935617919-15814-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:55:42.290723 4548 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:55:42.292002 4548 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:55:42.303064 4556 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:55:42.303184 4555 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:55:42.303208 4559 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:55:42.304047 4548 server_base.cc:1034] running on GCE node
I20260809 22:55:42.304780 4548 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:55:42.306051 4548 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:55:42.307241 4548 hybrid_clock.cc:648] HybridClock initialized: now 1786316142307193 us; error 45 us; skew 500 ppm
I20260809 22:55:42.307552 4548 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:55:42 dist-test-slave-ncp9 krb5kdc[4530](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316142, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.113.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:55:42.312810 4548 init.cc:377] Logged in from keytab as kudu/127.15.113.190@KRBTEST.COM (short username kudu)
I20260809 22:55:42.314232 4548 webserver.cc:460] Webserver started at http://127.15.113.190:41021/ using document root <none> and password file <none>
I20260809 22:55:42.314881 4548 fs_manager.cc:359] Metadata directory not provided
I20260809 22:55:42.314985 4548 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:55:42.315274 4548 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:55:42.318125 4548 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786315935617919-15814-0/minicluster-data/master-0/data/instance:
uuid: "0277478a7e37478c9e6d0c686f9236d6"
format_stamp: "Formatted at 2026-08-09 22:55:42 on dist-test-slave-ncp9"
server_key: "284217ab2f924135048f67029de9c13f"
server_key_iv: "4ffe4595907d3be48d5aae21ee707146"
server_key_version: "encryptionkey@0"
I20260809 22:55:42.318972 4548 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786315935617919-15814-0/minicluster-data/master-0/wal/instance:
uuid: "0277478a7e37478c9e6d0c686f9236d6"
format_stamp: "Formatted at 2026-08-09 22:55:42 on dist-test-slave-ncp9"
server_key: "284217ab2f924135048f67029de9c13f"
server_key_iv: "4ffe4595907d3be48d5aae21ee707146"
server_key_version: "encryptionkey@0"
I20260809 22:55:42.323798 4548 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.004s
I20260809 22:55:42.326995 4570 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:55:42.328226 4548 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260809 22:55:42.328377 4548 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786315935617919-15814-0/minicluster-data/master-0/data,/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786315935617919-15814-0/minicluster-data/master-0/wal
uuid: "0277478a7e37478c9e6d0c686f9236d6"
format_stamp: "Formatted at 2026-08-09 22:55:42 on dist-test-slave-ncp9"
server_key: "284217ab2f924135048f67029de9c13f"
server_key_iv: "4ffe4595907d3be48d5aae21ee707146"
server_key_version: "encryptionkey@0"
I20260809 22:55:42.328547 4548 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786315935617919-15814-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786315935617919-15814-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786315935617919-15814-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:55:42.345343 4548 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:55:42.348964 4548 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:55:42.349211 4548 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:55:42.366216 4548 rpc_server.cc:307] RPC server started. Bound to: 127.15.113.190:40961
I20260809 22:55:42.366261 4648 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.113.190:40961 every 8 connection(s)
I20260809 22:55:42.367451 4548 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786315935617919-15814-0/minicluster-data/master-0/data/info.pb
I20260809 22:55:42.368211 15814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskItYPJR/build/asan/bin/kudu as pid 4548
I20260809 22:55:42.368443 15814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786315935617919-15814-0/minicluster-data/master-0/wal/instance
I20260809 22:55:42.369171 15814 external_mini_cluster.cc:1428] Setting key 02683d8105b86b1f2ea54d28b7c3eb15
I20260809 22:55:42.371100 4649 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:55:42.377305 4649 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.113.190" port: 40961 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 22:55:42 dist-test-slave-ncp9 krb5kdc[4530](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316142, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.113.190@KRBTEST.COM for kudu/127.15.113.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.15.113.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.113.189@KRBTEST.COM" created.
I20260809 22:55:42.398075 4653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:55:42.381646 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56130 (local address 127.15.113.190:40961)
0809 22:55:42.382325 (+ 679us) server_negotiation.cc:206] Beginning negotiation
0809 22:55:42.382333 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 22:55:42.382379 (+ 46us) server_negotiation.cc:407] Connection header received
0809 22:55:42.383464 (+ 1085us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:55:42.383469 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:55:42.383677 (+ 208us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:55:42.383846 (+ 169us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:55:42.385083 (+ 1237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:55:42.386247 (+ 1164us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:55:42.387563 (+ 1316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:55:42.388071 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:55:42.388243 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:55:42.391684 (+ 3441us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:55:42.391692 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:55:42.391702 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:55:42.391730 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:55:42.394417 (+ 2687us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:55:42.395055 (+ 638us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:55:42.395061 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:55:42.395063 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:55:42.395304 (+ 241us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:55:42.395631 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:55:42.395635 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:55:42.395637 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:55:42.396098 (+ 461us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:55:42.396257 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0809 22:55:42.396854 (+ 597us) server_negotiation.cc:299] Negotiation successful
0809 22:55:42.397114 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":41,"server-negotiator.queue_time_us":461,"thread_start_us":281,"threads_started":1}
I20260809 22:55:42.398099 4652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:55:42.380423 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:40961 (local address 127.0.0.1:56130)
0809 22:55:42.381163 (+ 740us) negotiation.cc:107] Waiting for socket to connect
0809 22:55:42.381208 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0809 22:55:42.383145 (+ 1937us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:55:42.383885 (+ 740us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:55:42.383892 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:55:42.384537 (+ 645us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:55:42.384853 (+ 316us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:55:42.384870 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:55:42.386389 (+ 1519us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:55:42.386394 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:55:42.387436 (+ 1042us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:55:42.387448 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:55:42.388282 (+ 834us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:55:42.388289 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:55:42.388548 (+ 259us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:55:42.389147 (+ 599us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:55:42.389169 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:55:42.391456 (+ 2287us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:55:42.394532 (+ 3076us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:55:42.394542 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:55:42.394550 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:55:42.394942 (+ 392us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:55:42.395397 (+ 455us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:55:42.395400 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:55:42.395402 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:55:42.395545 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:55:42.396260 (+ 715us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:55:42.396269 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:55:42.396495 (+ 226us) client_negotiation.cc:769] Sending connection context
0809 22:55:42.396735 (+ 240us) client_negotiation.cc:240] Negotiation successful
0809 22:55:42.397281 (+ 546us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":518,"mutex_wait_us":118,"thread_start_us":305,"threads_started":1}
I20260809 22:55:42.400981 4649 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.113.189" port: 38369 } has no permanent_uuid. Determining permanent_uuid...
I20260809 22:55:42.401940 4652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:55:42.401612 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.189:38369 (local address 127.0.0.1:40774)
0809 22:55:42.401754 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0809 22:55:42.401849 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.113.189:38369: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260809 22:55:42.402245 4573 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.15.113.189:38369: connect: Connection refused (error 111)
I20260809 22:55:42.403678 4652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:55:42.403361 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.189:38369 (local address 127.0.0.1:40784)
0809 22:55:42.403499 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0809 22:55:42.403561 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.113.189:38369: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260809 22:55:42.403867 4649 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.113.189:38369: Network error: Client connection negotiation failed: client connection to 127.15.113.189:38369: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.113.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.189.keytab.
Entry for principal kudu/127.15.113.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.189.keytab.
I20260809 22:55:42.424036 15814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskItYPJR/build/asan/bin/kudu
/tmp/dist-test-taskItYPJR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786315935617919-15814-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786315935617919-15814-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786315935617919-15814-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786315935617919-15814-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.113.189:38369
--webserver_interface=127.15.113.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.189.keytab
--principal=kudu/127.15.113.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.113.148:44461
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.113.190:40961,127.15.113.189:38369,127.15.113.188:39007
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786315935617919-15814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786315935617919-15814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786315935617919-15814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 22:55:42.453135 4649 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.113.189" port: 38369 } attempt: 1
I20260809 22:55:42.454244 4652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:55:42.453824 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.189:38369 (local address 127.0.0.1:40790)
0809 22:55:42.454003 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0809 22:55:42.454120 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.113.189:38369: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260809 22:55:42.455145 4652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:55:42.454839 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.189:38369 (local address 127.0.0.1:40806)
0809 22:55:42.454964 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0809 22:55:42.455027 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.113.189:38369: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260809 22:55:42.455338 4649 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.113.189:38369: Network error: Client connection negotiation failed: client connection to 127.15.113.189:38369: connect: Connection refused (error 111)
I20260809 22:55:42.523571 4649 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.113.189" port: 38369 } attempt: 2
I20260809 22:55:42.524735 4652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:55:42.524273 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.189:38369 (local address 127.0.0.1:40816)
0809 22:55:42.524498 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0809 22:55:42.524590 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.113.189:38369: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
I20260809 22:55:42.525734 4652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:55:42.525403 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.189:38369 (local address 127.0.0.1:40818)
0809 22:55:42.525543 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0809 22:55:42.525606 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.113.189:38369: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260809 22:55:42.525920 4649 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.113.189:38369: Network error: Client connection negotiation failed: client connection to 127.15.113.189:38369: connect: Connection refused (error 111)
I20260809 22:55:42.617136 4649 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.113.189" port: 38369 } attempt: 3
I20260809 22:55:42.618353 4652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:55:42.617914 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.189:38369 (local address 127.0.0.1:40824)
0809 22:55:42.618115 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0809 22:55:42.618221 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.113.189:38369: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
I20260809 22:55:42.619354 4652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:55:42.619030 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.189:38369 (local address 127.0.0.1:40826)
0809 22:55:42.619162 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0809 22:55:42.619228 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.113.189:38369: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260809 22:55:42.619544 4649 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.113.189:38369: Network error: Client connection negotiation failed: client connection to 127.15.113.189:38369: connect: Connection refused (error 111)
W20260809 22:55:42.750147 4658 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:55:42.750422 4658 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:55:42.750515 4658 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:55:42.757469 4658 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 22:55:42.757563 4658 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:55:42.757615 4658 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:55:42.757670 4658 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 22:55:42.757712 4658 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 22:55:42.762759 4649 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.113.189" port: 38369 } attempt: 4
I20260809 22:55:42.763935 4652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:55:42.763401 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.189:38369 (local address 127.0.0.1:40838)
0809 22:55:42.763716 (+ 315us) negotiation.cc:107] Waiting for socket to connect
0809 22:55:42.763808 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.113.189:38369: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":166}
I20260809 22:55:42.764910 4652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:55:42.764556 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.189:38369 (local address 127.0.0.1:40852)
0809 22:55:42.764694 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0809 22:55:42.764779 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.113.189:38369: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260809 22:55:42.765069 4649 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.113.189:38369: Network error: Client connection negotiation failed: client connection to 127.15.113.189:38369: connect: Connection refused (error 111)
I20260809 22:55:42.767550 4658 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.113.148:44461
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786315935617919-15814-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786315935617919-15814-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.15.113.190:40961,127.15.113.189:38369,127.15.113.188:39007
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.189.keytab
--principal=kudu/127.15.113.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.113.189:38369
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786315935617919-15814-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.15.113.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786315935617919-15814-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:55:42.768769 4658 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:55:42.770032 4658 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:55:42.781191 4665 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:55:42.781548 4669 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:55:42.781586 4667 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:55:42.782411 4658 server_base.cc:1034] running on GCE node
I20260809 22:55:42.783051 4658 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:55:42.784257 4658 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:55:42.785431 4658 hybrid_clock.cc:648] HybridClock initialized: now 1786316142785382 us; error 43 us; skew 500 ppm
I20260809 22:55:42.785732 4658 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:55:42 dist-test-slave-ncp9 krb5kdc[4530](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316142, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.113.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:55:42.788625 4658 init.cc:377] Logged in from keytab as kudu/127.15.113.189@KRBTEST.COM (short username kudu)
I20260809 22:55:42.789942 4658 webserver.cc:460] Webserver started at http://127.15.113.189:42919/ using document root <none> and password file <none>
I20260809 22:55:42.790556 4658 fs_manager.cc:359] Metadata directory not provided
I20260809 22:55:42.790664 4658 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:55:42.790939 4658 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:55:42.793792 4658 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786315935617919-15814-0/minicluster-data/master-1/data/instance:
uuid: "9bc5f2accd184ca987d8d463e8544c61"
format_stamp: "Formatted at 2026-08-09 22:55:42 on dist-test-slave-ncp9"
server_key: "7fff7203f8d24e1d324c0347ffe2ebdf"
server_key_iv: "f54f9a7c57ea2a1bb8dfac1f62da7197"
server_key_version: "encryptionkey@0"
I20260809 22:55:42.794546 4658 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786315935617919-15814-0/minicluster-data/master-1/wal/instance:
uuid: "9bc5f2accd184ca987d8d463e8544c61"
format_stamp: "Formatted at 2026-08-09 22:55:42 on dist-test-slave-ncp9"
server_key: "7fff7203f8d24e1d324c0347ffe2ebdf"
server_key_iv: "f54f9a7c57ea2a1bb8dfac1f62da7197"
server_key_version: "encryptionkey@0"
I20260809 22:55:42.799292 4658 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260809 22:55:42.802489 4677 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:55:42.803664 4658 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260809 22:55:42.803817 4658 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786315935617919-15814-0/minicluster-data/master-1/data,/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786315935617919-15814-0/minicluster-data/master-1/wal
uuid: "9bc5f2accd184ca987d8d463e8544c61"
format_stamp: "Formatted at 2026-08-09 22:55:42 on dist-test-slave-ncp9"
server_key: "7fff7203f8d24e1d324c0347ffe2ebdf"
server_key_iv: "f54f9a7c57ea2a1bb8dfac1f62da7197"
server_key_version: "encryptionkey@0"
I20260809 22:55:42.803985 4658 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786315935617919-15814-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786315935617919-15814-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786315935617919-15814-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:55:42.830868 4658 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:55:42.834905 4658 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:55:42.835160 4658 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:55:42.851672 4658 rpc_server.cc:307] RPC server started. Bound to: 127.15.113.189:38369
I20260809 22:55:42.851713 4754 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.113.189:38369 every 8 connection(s)
I20260809 22:55:42.852919 4658 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786315935617919-15814-0/minicluster-data/master-1/data/info.pb
I20260809 22:55:42.855998 15814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskItYPJR/build/asan/bin/kudu as pid 4658
I20260809 22:55:42.856156 15814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786315935617919-15814-0/minicluster-data/master-1/wal/instance
I20260809 22:55:42.856521 4755 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:55:42.856607 15814 external_mini_cluster.cc:1428] Setting key 55d55829d2f864371866296dd5c8c1f5
I20260809 22:55:42.862829 4755 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.113.190" port: 40961 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 22:55:42 dist-test-slave-ncp9 krb5kdc[4530](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316142, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.113.189@KRBTEST.COM for kudu/127.15.113.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.15.113.188@KRBTEST.COM; defaulting to no policy
I20260809 22:55:42.882629 4653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:55:42.865282 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56146 (local address 127.15.113.190:40961)
0809 22:55:42.865485 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0809 22:55:42.865491 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:55:42.866634 (+ 1143us) server_negotiation.cc:407] Connection header received
0809 22:55:42.868449 (+ 1815us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:55:42.868463 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:55:42.868548 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:55:42.868669 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:55:42.870079 (+ 1410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:55:42.871063 (+ 984us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:55:42.872522 (+ 1459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:55:42.873157 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:55:42.873289 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:55:42.876409 (+ 3120us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:55:42.876415 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:55:42.876419 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:55:42.876465 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:55:42.879661 (+ 3196us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:55:42.880342 (+ 681us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:55:42.880348 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:55:42.880352 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:55:42.880514 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:55:42.880876 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:55:42.880881 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:55:42.880884 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:55:42.881286 (+ 402us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:55:42.881446 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0809 22:55:42.882036 (+ 590us) server_negotiation.cc:299] Negotiation successful
0809 22:55:42.882274 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260809 22:55:42.883247 4759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:55:42.865716 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:40961 (local address 127.0.0.1:56146)
0809 22:55:42.866461 (+ 745us) negotiation.cc:107] Waiting for socket to connect
0809 22:55:42.866505 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 22:55:42.868185 (+ 1680us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:55:42.868834 (+ 649us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:55:42.868842 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:55:42.869355 (+ 513us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:55:42.869884 (+ 529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:55:42.869902 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:55:42.871193 (+ 1291us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:55:42.871197 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:55:42.872373 (+ 1176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:55:42.872385 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:55:42.873345 (+ 960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:55:42.873350 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:55:42.873530 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:55:42.874135 (+ 605us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:55:42.874155 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:55:42.876193 (+ 2038us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:55:42.879796 (+ 3603us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:55:42.879803 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:55:42.879813 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:55:42.880230 (+ 417us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:55:42.880609 (+ 379us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:55:42.880612 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:55:42.880614 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:55:42.880775 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:55:42.881401 (+ 626us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:55:42.881406 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:55:42.881683 (+ 277us) client_negotiation.cc:769] Sending connection context
0809 22:55:42.881899 (+ 216us) client_negotiation.cc:240] Negotiation successful
0809 22:55:42.882258 (+ 359us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":491,"thread_start_us":322,"threads_started":1}
I20260809 22:55:42.884984 4755 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.113.189" port: 38369 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.113.188@KRBTEST.COM" created.
Aug 09 22:55:42 dist-test-slave-ncp9 krb5kdc[4530](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316142, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.113.189@KRBTEST.COM for kudu/127.15.113.189@KRBTEST.COM
I20260809 22:55:42.899261 4759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:55:42.885728 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.189:38369 (local address 127.0.0.1:40862)
0809 22:55:42.885912 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0809 22:55:42.885929 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 22:55:42.886046 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:55:42.887090 (+ 1044us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:55:42.887093 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:55:42.887411 (+ 318us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:55:42.887534 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:55:42.887541 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:55:42.889262 (+ 1721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:55:42.889265 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:55:42.890060 (+ 795us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:55:42.890070 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:55:42.891060 (+ 990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:55:42.891064 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:55:42.891235 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:55:42.891733 (+ 498us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:55:42.891747 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:55:42.893740 (+ 1993us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:55:42.896934 (+ 3194us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:55:42.896939 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:55:42.896943 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:55:42.897294 (+ 351us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:55:42.897652 (+ 358us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:55:42.897657 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:55:42.897661 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:55:42.897788 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:55:42.898382 (+ 594us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:55:42.898387 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:55:42.898509 (+ 122us) client_negotiation.cc:769] Sending connection context
0809 22:55:42.898630 (+ 121us) client_negotiation.cc:240] Negotiation successful
0809 22:55:42.898880 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":63,"mutex_wait_us":2}
I20260809 22:55:42.899345 4761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:55:42.885930 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40862 (local address 127.15.113.189:38369)
0809 22:55:42.886631 (+ 701us) server_negotiation.cc:206] Beginning negotiation
0809 22:55:42.886638 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:55:42.886668 (+ 30us) server_negotiation.cc:407] Connection header received
0809 22:55:42.886724 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:55:42.886732 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:55:42.886938 (+ 206us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:55:42.887086 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:55:42.887680 (+ 594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:55:42.889141 (+ 1461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:55:42.890195 (+ 1054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:55:42.890935 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:55:42.891052 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:55:42.893960 (+ 2908us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:55:42.893975 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:55:42.893990 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:55:42.894023 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:55:42.896775 (+ 2752us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:55:42.897414 (+ 639us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:55:42.897420 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:55:42.897422 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:55:42.897533 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:55:42.897886 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:55:42.897891 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:55:42.897894 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:55:42.898225 (+ 331us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:55:42.898413 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0809 22:55:42.898776 (+ 363us) server_negotiation.cc:299] Negotiation successful
0809 22:55:42.898978 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":41,"server-negotiator.queue_time_us":532,"thread_start_us":392,"threads_started":1}
I20260809 22:55:42.901801 4755 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.113.188" port: 39007 } has no permanent_uuid. Determining permanent_uuid...
I20260809 22:55:42.902892 4759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:55:42.902406 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.188:39007 (local address 127.0.0.1:52932)
0809 22:55:42.902645 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0809 22:55:42.902745 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.113.188:39007: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260809 22:55:42.903111 4682 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.15.113.188:39007: connect: Connection refused (error 111)
I20260809 22:55:42.904574 4759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:55:42.904271 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.188:39007 (local address 127.0.0.1:52940)
0809 22:55:42.904404 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0809 22:55:42.904476 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.113.188:39007: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260809 22:55:42.904783 4755 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.113.188:39007: Network error: Client connection negotiation failed: client connection to 127.15.113.188:39007: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.113.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.188.keytab.
Entry for principal kudu/127.15.113.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.188.keytab.
I20260809 22:55:42.918735 15814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskItYPJR/build/asan/bin/kudu
/tmp/dist-test-taskItYPJR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786315935617919-15814-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786315935617919-15814-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786315935617919-15814-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786315935617919-15814-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.113.188:39007
--webserver_interface=127.15.113.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.188.keytab
--principal=kudu/127.15.113.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.113.148:44461
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.113.190:40961,127.15.113.189:38369,127.15.113.188:39007
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786315935617919-15814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786315935617919-15814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786315935617919-15814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 22:55:42.954075 4755 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.113.188" port: 39007 } attempt: 1
I20260809 22:55:42.955123 4759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:55:42.954687 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.188:39007 (local address 127.0.0.1:52952)
0809 22:55:42.954873 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0809 22:55:42.954974 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.113.188:39007: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260809 22:55:42.956081 4759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:55:42.955771 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.188:39007 (local address 127.0.0.1:52960)
0809 22:55:42.955912 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0809 22:55:42.955977 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.113.188:39007: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260809 22:55:42.956281 4755 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.113.188:39007: Network error: Client connection negotiation failed: client connection to 127.15.113.188:39007: connect: Connection refused (error 111)
I20260809 22:55:43.024467 4755 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.113.188" port: 39007 } attempt: 2
I20260809 22:55:43.025545 4759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:55:43.025101 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.188:39007 (local address 127.0.0.1:52974)
0809 22:55:43.025302 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0809 22:55:43.025391 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.113.188:39007: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
I20260809 22:55:43.026558 4759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:55:43.026224 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.188:39007 (local address 127.0.0.1:52986)
0809 22:55:43.026368 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0809 22:55:43.026436 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.113.188:39007: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260809 22:55:43.026765 4755 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.113.188:39007: Network error: Client connection negotiation failed: client connection to 127.15.113.188:39007: connect: Connection refused (error 111)
I20260809 22:55:43.064312 4649 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.113.189" port: 38369 } attempt: 5
Aug 09 22:55:43 dist-test-slave-ncp9 krb5kdc[4530](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316142, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.113.190@KRBTEST.COM for kudu/127.15.113.189@KRBTEST.COM
I20260809 22:55:43.077237 4652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:55:43.065106 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.189:38369 (local address 127.0.0.1:40866)
0809 22:55:43.065341 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0809 22:55:43.065361 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 22:55:43.065501 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:55:43.065857 (+ 356us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:55:43.065862 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:55:43.066263 (+ 401us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:55:43.066454 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:55:43.066463 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:55:43.067473 (+ 1010us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:55:43.067479 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:55:43.068367 (+ 888us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:55:43.068380 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:55:43.069187 (+ 807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:55:43.069193 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:55:43.069374 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:55:43.069956 (+ 582us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:55:43.069974 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:55:43.071975 (+ 2001us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:55:43.074897 (+ 2922us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:55:43.074903 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:55:43.074908 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:55:43.075267 (+ 359us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:55:43.075612 (+ 345us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:55:43.075617 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:55:43.075630 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:55:43.075767 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:55:43.076280 (+ 513us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:55:43.076284 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:55:43.076431 (+ 147us) client_negotiation.cc:769] Sending connection context
0809 22:55:43.076554 (+ 123us) client_negotiation.cc:240] Negotiation successful
0809 22:55:43.076846 (+ 292us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111}
I20260809 22:55:43.077239 4761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:55:43.065129 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40866 (local address 127.15.113.189:38369)
0809 22:55:43.065314 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0809 22:55:43.065319 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:55:43.065440 (+ 121us) server_negotiation.cc:407] Connection header received
0809 22:55:43.065644 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:55:43.065648 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:55:43.065716 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:55:43.065804 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:55:43.066633 (+ 829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:55:43.067337 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:55:43.068534 (+ 1197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:55:43.069041 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:55:43.069163 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:55:43.072167 (+ 3004us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:55:43.072172 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:55:43.072175 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:55:43.072208 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:55:43.074758 (+ 2550us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:55:43.075380 (+ 622us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:55:43.075385 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:55:43.075387 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:55:43.075506 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:55:43.075870 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:55:43.075875 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:55:43.075878 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:55:43.076164 (+ 286us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:55:43.076295 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0809 22:55:43.076682 (+ 387us) server_negotiation.cc:299] Negotiation successful
0809 22:55:43.076885 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260809 22:55:43.078528 4649 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.113.188" port: 39007 } has no permanent_uuid. Determining permanent_uuid...
I20260809 22:55:43.079664 4652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:55:43.079281 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.188:39007 (local address 127.0.0.1:52998)
0809 22:55:43.079448 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0809 22:55:43.079524 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.113.188:39007: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260809 22:55:43.080665 4652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:55:43.080352 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.188:39007 (local address 127.0.0.1:53004)
0809 22:55:43.080489 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0809 22:55:43.080554 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.113.188:39007: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260809 22:55:43.080869 4649 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.113.188:39007: Network error: Client connection negotiation failed: client connection to 127.15.113.188:39007: connect: Connection refused (error 111)
I20260809 22:55:43.117952 4755 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.113.188" port: 39007 } attempt: 3
I20260809 22:55:43.118975 4759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:55:43.118532 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.188:39007 (local address 127.0.0.1:53006)
0809 22:55:43.118728 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0809 22:55:43.118817 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.113.188:39007: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260809 22:55:43.120079 4759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:55:43.119666 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.188:39007 (local address 127.0.0.1:53022)
0809 22:55:43.119886 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0809 22:55:43.119967 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.113.188:39007: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":139}
W20260809 22:55:43.120288 4755 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.113.188:39007: Network error: Client connection negotiation failed: client connection to 127.15.113.188:39007: connect: Connection refused (error 111)
I20260809 22:55:43.132090 4649 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.113.188" port: 39007 } attempt: 1
I20260809 22:55:43.133081 4652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:55:43.132644 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.188:39007 (local address 127.0.0.1:53028)
0809 22:55:43.132804 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0809 22:55:43.132920 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.113.188:39007: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260809 22:55:43.133989 4652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:55:43.133699 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.188:39007 (local address 127.0.0.1:53030)
0809 22:55:43.133826 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0809 22:55:43.133893 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.113.188:39007: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260809 22:55:43.134196 4649 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.113.188:39007: Network error: Client connection negotiation failed: client connection to 127.15.113.188:39007: connect: Connection refused (error 111)
I20260809 22:55:43.202386 4649 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.113.188" port: 39007 } attempt: 2
I20260809 22:55:43.203533 4652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:55:43.203082 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.188:39007 (local address 127.0.0.1:53046)
0809 22:55:43.203282 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0809 22:55:43.203371 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.113.188:39007: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260809 22:55:43.204632 4652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:55:43.204176 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.188:39007 (local address 127.0.0.1:53052)
0809 22:55:43.204385 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0809 22:55:43.204489 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.113.188:39007: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260809 22:55:43.204959 4649 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.113.188:39007: Network error: Client connection negotiation failed: client connection to 127.15.113.188:39007: connect: Connection refused (error 111)
W20260809 22:55:43.247066 4764 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:55:43.247359 4764 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:55:43.247504 4764 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:55:43.254654 4764 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 22:55:43.254770 4764 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:55:43.254846 4764 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:55:43.254925 4764 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 22:55:43.254988 4764 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 22:55:43.263473 4755 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.113.188" port: 39007 } attempt: 4
I20260809 22:55:43.264596 4759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:55:43.264131 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.188:39007 (local address 127.0.0.1:53054)
0809 22:55:43.264354 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0809 22:55:43.264452 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.113.188:39007: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260809 22:55:43.264747 4764 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.113.148:44461
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786315935617919-15814-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786315935617919-15814-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.15.113.190:40961,127.15.113.189:38369,127.15.113.188:39007
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.188.keytab
--principal=kudu/127.15.113.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.113.188:39007
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786315935617919-15814-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.15.113.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786315935617919-15814-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:55:43.265555 4759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:55:43.265238 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.188:39007 (local address 127.0.0.1:53068)
0809 22:55:43.265384 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0809 22:55:43.265454 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.113.188:39007: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260809 22:55:43.265741 4755 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.113.188:39007: Network error: Client connection negotiation failed: client connection to 127.15.113.188:39007: connect: Connection refused (error 111)
I20260809 22:55:43.266096 4764 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:55:43.267343 4764 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:55:43.278575 4772 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:55:43.278705 4775 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:55:43.278728 4771 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:55:43.279353 4764 server_base.cc:1034] running on GCE node
I20260809 22:55:43.280073 4764 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:55:43.281390 4764 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:55:43.282601 4764 hybrid_clock.cc:648] HybridClock initialized: now 1786316143282547 us; error 42 us; skew 500 ppm
I20260809 22:55:43.282927 4764 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:55:43 dist-test-slave-ncp9 krb5kdc[4530](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316143, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.113.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:55:43.285833 4764 init.cc:377] Logged in from keytab as kudu/127.15.113.188@KRBTEST.COM (short username kudu)
I20260809 22:55:43.287118 4764 webserver.cc:460] Webserver started at http://127.15.113.188:36119/ using document root <none> and password file <none>
I20260809 22:55:43.287720 4764 fs_manager.cc:359] Metadata directory not provided
I20260809 22:55:43.287817 4764 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:55:43.288056 4764 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:55:43.290863 4764 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786315935617919-15814-0/minicluster-data/master-2/data/instance:
uuid: "c64aba6229c14a26ac312e768826845f"
format_stamp: "Formatted at 2026-08-09 22:55:43 on dist-test-slave-ncp9"
server_key: "87a2819c2f5aaf48a57d9e82670ec341"
server_key_iv: "5b287fca430454f16c014ac1a0d8def1"
server_key_version: "encryptionkey@0"
I20260809 22:55:43.291632 4764 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786315935617919-15814-0/minicluster-data/master-2/wal/instance:
uuid: "c64aba6229c14a26ac312e768826845f"
format_stamp: "Formatted at 2026-08-09 22:55:43 on dist-test-slave-ncp9"
server_key: "87a2819c2f5aaf48a57d9e82670ec341"
server_key_iv: "5b287fca430454f16c014ac1a0d8def1"
server_key_version: "encryptionkey@0"
I20260809 22:55:43.296427 4764 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.001s
I20260809 22:55:43.299727 4787 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:55:43.300926 4764 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 22:55:43.301100 4764 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786315935617919-15814-0/minicluster-data/master-2/data,/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786315935617919-15814-0/minicluster-data/master-2/wal
uuid: "c64aba6229c14a26ac312e768826845f"
format_stamp: "Formatted at 2026-08-09 22:55:43 on dist-test-slave-ncp9"
server_key: "87a2819c2f5aaf48a57d9e82670ec341"
server_key_iv: "5b287fca430454f16c014ac1a0d8def1"
server_key_version: "encryptionkey@0"
I20260809 22:55:43.301260 4764 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786315935617919-15814-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786315935617919-15814-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786315935617919-15814-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:55:43.311172 4649 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.113.188" port: 39007 } attempt: 3
I20260809 22:55:43.312180 4652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:55:43.311698 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.188:39007 (local address 127.0.0.1:53074)
0809 22:55:43.311913 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0809 22:55:43.312006 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.113.188:39007: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260809 22:55:43.313117 4652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:55:43.312725 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.188:39007 (local address 127.0.0.1:53078)
0809 22:55:43.312918 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0809 22:55:43.313000 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.113.188:39007: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
W20260809 22:55:43.313310 4649 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.113.188:39007: Network error: Client connection negotiation failed: client connection to 127.15.113.188:39007: connect: Connection refused (error 111)
I20260809 22:55:43.327122 4764 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:55:43.330778 4764 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:55:43.331022 4764 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:55:43.348049 4764 rpc_server.cc:307] RPC server started. Bound to: 127.15.113.188:39007
I20260809 22:55:43.348078 4854 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.113.188:39007 every 8 connection(s)
I20260809 22:55:43.349291 4764 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786315935617919-15814-0/minicluster-data/master-2/data/info.pb
I20260809 22:55:43.352246 15814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskItYPJR/build/asan/bin/kudu as pid 4764
I20260809 22:55:43.352428 15814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786315935617919-15814-0/minicluster-data/master-2/wal/instance
I20260809 22:55:43.353022 15814 external_mini_cluster.cc:1428] Setting key ad88abb6057085628f57b4a84d24e96b
I20260809 22:55:43.353109 4855 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:55:43.362246 4855 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.113.190" port: 40961 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 22:55:43 dist-test-slave-ncp9 krb5kdc[4530](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316141, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.113.188@KRBTEST.COM
Aug 09 22:55:43 dist-test-slave-ncp9 krb5kdc[4530](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316143, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.113.188@KRBTEST.COM for kudu/127.15.113.190@KRBTEST.COM
I20260809 22:55:43.379125 4653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:55:43.365214 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56150 (local address 127.15.113.190:40961)
0809 22:55:43.365457 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0809 22:55:43.365465 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 22:55:43.366081 (+ 616us) server_negotiation.cc:407] Connection header received
0809 22:55:43.366272 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:55:43.366278 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:55:43.366349 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:55:43.366433 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:55:43.367488 (+ 1055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:55:43.368174 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:55:43.369527 (+ 1353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:55:43.370236 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:55:43.370366 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:55:43.373776 (+ 3410us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:55:43.373782 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:55:43.373786 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:55:43.373822 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:55:43.376369 (+ 2547us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:55:43.376995 (+ 626us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:55:43.377000 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:55:43.377004 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:55:43.377135 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:55:43.377515 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:55:43.377520 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:55:43.377523 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:55:43.377780 (+ 257us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:55:43.377892 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0809 22:55:43.378550 (+ 658us) server_negotiation.cc:299] Negotiation successful
0809 22:55:43.378785 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260809 22:55:43.380163 4860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:55:43.365127 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:40961 (local address 127.0.0.1:56150)
0809 22:55:43.365959 (+ 832us) negotiation.cc:107] Waiting for socket to connect
0809 22:55:43.365993 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0809 22:55:43.366151 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:55:43.366510 (+ 359us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:55:43.366517 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:55:43.367092 (+ 575us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:55:43.367346 (+ 254us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:55:43.367355 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:55:43.368328 (+ 973us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:55:43.368333 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:55:43.369380 (+ 1047us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:55:43.369396 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:55:43.370484 (+ 1088us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:55:43.370492 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:55:43.370674 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:55:43.371426 (+ 752us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:55:43.371456 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:55:43.373572 (+ 2116us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:55:43.376488 (+ 2916us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:55:43.376497 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:55:43.376504 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:55:43.376871 (+ 367us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:55:43.377238 (+ 367us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:55:43.377242 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:55:43.377246 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:55:43.377402 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:55:43.377899 (+ 497us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:55:43.377906 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:55:43.378127 (+ 221us) client_negotiation.cc:769] Sending connection context
0809 22:55:43.378414 (+ 287us) client_negotiation.cc:240] Negotiation successful
0809 22:55:43.378859 (+ 445us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":603,"spinlock_wait_cycles":29696,"thread_start_us":493,"threads_started":1}
I20260809 22:55:43.380183 4859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:55:43.354910 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53094 (local address 127.15.113.188:39007)
0809 22:55:43.355680 (+ 770us) server_negotiation.cc:206] Beginning negotiation
0809 22:55:43.355691 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 22:55:43.355732 (+ 41us) server_negotiation.cc:407] Connection header received
0809 22:55:43.357633 (+ 1901us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:55:43.357655 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:55:43.358023 (+ 368us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:55:43.358351 (+ 328us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:55:43.360153 (+ 1802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:55:43.361486 (+ 1333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:55:43.363446 (+ 1960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:55:43.364438 (+ 992us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:55:43.364628 (+ 190us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:55:43.373388 (+ 8760us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:55:43.373400 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:55:43.373416 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:55:43.373458 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:55:43.376186 (+ 2728us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:55:43.377002 (+ 816us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:55:43.377010 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:55:43.377012 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:55:43.377172 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:55:43.377531 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:55:43.377535 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:55:43.377537 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:55:43.378077 (+ 540us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:55:43.378359 (+ 282us) server_negotiation.cc:1035] Waiting for connection context
0809 22:55:43.378652 (+ 293us) server_negotiation.cc:299] Negotiation successful
0809 22:55:43.379155 (+ 503us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":113,"server-negotiator.queue_time_us":578,"thread_start_us":340,"threads_started":1}
I20260809 22:55:43.381913 4855 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.113.189" port: 38369 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 22:55:43 dist-test-slave-ncp9 krb5kdc[4530](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316143, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.113.188@KRBTEST.COM for kudu/127.15.113.189@KRBTEST.COM
Aug 09 22:55:43 dist-test-slave-ncp9 krb5kdc[4530](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316141, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.113.190@KRBTEST.COM
I20260809 22:55:43.395169 4761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:55:43.382647 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40874 (local address 127.15.113.189:38369)
0809 22:55:43.382844 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0809 22:55:43.382850 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:55:43.383208 (+ 358us) server_negotiation.cc:407] Connection header received
0809 22:55:43.383279 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:55:43.383284 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:55:43.383360 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:55:43.383450 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:55:43.384057 (+ 607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:55:43.385075 (+ 1018us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:55:43.386176 (+ 1101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:55:43.386812 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:55:43.386933 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:55:43.389740 (+ 2807us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:55:43.389747 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:55:43.389752 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:55:43.389792 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:55:43.392780 (+ 2988us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:55:43.393501 (+ 721us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:55:43.393505 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:55:43.393507 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:55:43.393638 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:55:43.394024 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:55:43.394027 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:55:43.394029 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:55:43.394270 (+ 241us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:55:43.394381 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0809 22:55:43.394767 (+ 386us) server_negotiation.cc:299] Negotiation successful
0809 22:55:43.394914 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260809 22:55:43.395285 4860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:55:43.382686 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.189:38369 (local address 127.0.0.1:40874)
0809 22:55:43.382968 (+ 282us) negotiation.cc:107] Waiting for socket to connect
0809 22:55:43.382986 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 22:55:43.383085 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:55:43.383498 (+ 413us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:55:43.383501 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:55:43.383791 (+ 290us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:55:43.383928 (+ 137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:55:43.383935 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:55:43.385207 (+ 1272us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:55:43.385211 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:55:43.386068 (+ 857us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:55:43.386079 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:55:43.386928 (+ 849us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:55:43.386932 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:55:43.387081 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:55:43.387538 (+ 457us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:55:43.387561 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:55:43.389530 (+ 1969us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:55:43.392931 (+ 3401us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:55:43.392936 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:55:43.392940 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:55:43.393391 (+ 451us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:55:43.393740 (+ 349us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:55:43.393744 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:55:43.393748 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:55:43.393908 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:55:43.394385 (+ 477us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:55:43.394390 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:55:43.394559 (+ 169us) client_negotiation.cc:769] Sending connection context
0809 22:55:43.394708 (+ 149us) client_negotiation.cc:240] Negotiation successful
0809 22:55:43.394965 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":184}
I20260809 22:55:43.396258 4855 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.113.188" port: 39007 } has no permanent_uuid. Determining permanent_uuid...
I20260809 22:55:43.397991 4653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:55:43.383129 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56164 (local address 127.15.113.190:40961)
0809 22:55:43.383649 (+ 520us) server_negotiation.cc:206] Beginning negotiation
0809 22:55:43.383653 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 22:55:43.383665 (+ 12us) server_negotiation.cc:407] Connection header received
0809 22:55:43.383699 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:55:43.383702 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:55:43.383747 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:55:43.383823 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:55:43.385289 (+ 1466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:55:43.385990 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:55:43.387551 (+ 1561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:55:43.387935 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:55:43.388043 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:55:43.392489 (+ 4446us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:55:43.392493 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:55:43.392496 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:55:43.392524 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:55:43.395517 (+ 2993us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:55:43.396127 (+ 610us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:55:43.396133 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:55:43.396149 (+ 16us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:55:43.396310 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:55:43.396702 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:55:43.396705 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:55:43.396707 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:55:43.397003 (+ 296us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:55:43.397244 (+ 241us) server_negotiation.cc:1035] Waiting for connection context
0809 22:55:43.397510 (+ 266us) server_negotiation.cc:299] Negotiation successful
0809 22:55:43.397709 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":423}
Aug 09 22:55:43 dist-test-slave-ncp9 krb5kdc[4530](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316143, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.113.188@KRBTEST.COM for kudu/127.15.113.188@KRBTEST.COM
Aug 09 22:55:43 dist-test-slave-ncp9 krb5kdc[4530](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316141, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.113.189@KRBTEST.COM
I20260809 22:55:43.409682 4860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:55:43.397026 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.188:39007 (local address 127.0.0.1:53102)
0809 22:55:43.397206 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0809 22:55:43.397223 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 22:55:43.397307 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:55:43.397604 (+ 297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:55:43.397608 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:55:43.397973 (+ 365us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:55:43.398143 (+ 170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:55:43.398154 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:55:43.399054 (+ 900us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:55:43.399058 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:55:43.400208 (+ 1150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:55:43.400220 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:55:43.401123 (+ 903us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:55:43.401127 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:55:43.401260 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:55:43.401723 (+ 463us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:55:43.401744 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:55:43.403952 (+ 2208us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:55:43.407501 (+ 3549us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:55:43.407505 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:55:43.407507 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:55:43.407868 (+ 361us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:55:43.408232 (+ 364us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:55:43.408236 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:55:43.408239 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:55:43.408382 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:55:43.408878 (+ 496us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:55:43.408882 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:55:43.409087 (+ 205us) client_negotiation.cc:769] Sending connection context
0809 22:55:43.409228 (+ 141us) client_negotiation.cc:240] Negotiation successful
0809 22:55:43.409416 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":66}
I20260809 22:55:43.409929 4859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:55:43.397262 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53102 (local address 127.15.113.188:39007)
0809 22:55:43.397406 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0809 22:55:43.397411 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:55:43.397425 (+ 14us) server_negotiation.cc:407] Connection header received
0809 22:55:43.397458 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:55:43.397461 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:55:43.397505 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:55:43.397576 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:55:43.398262 (+ 686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:55:43.398945 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:55:43.400341 (+ 1396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:55:43.400998 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:55:43.401120 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:55:43.404139 (+ 3019us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:55:43.404146 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:55:43.404151 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:55:43.404182 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:55:43.407388 (+ 3206us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:55:43.407965 (+ 577us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:55:43.407970 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:55:43.407973 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:55:43.408094 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:55:43.408498 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:55:43.408502 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:55:43.408504 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:55:43.408755 (+ 251us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:55:43.408858 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0809 22:55:43.409340 (+ 482us) server_negotiation.cc:299] Negotiation successful
0809 22:55:43.409586 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260809 22:55:43.414570 4761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:55:43.399534 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40890 (local address 127.15.113.189:38369)
0809 22:55:43.399707 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0809 22:55:43.399712 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:55:43.399729 (+ 17us) server_negotiation.cc:407] Connection header received
0809 22:55:43.399772 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:55:43.399777 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:55:43.399837 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:55:43.399930 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:55:43.401353 (+ 1423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:55:43.402388 (+ 1035us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:55:43.403874 (+ 1486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:55:43.404403 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:55:43.404486 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:55:43.409158 (+ 4672us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:55:43.409162 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:55:43.409165 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:55:43.409193 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:55:43.412192 (+ 2999us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:55:43.412791 (+ 599us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:55:43.412796 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:55:43.412800 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:55:43.412970 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:55:43.413290 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:55:43.413294 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:55:43.413298 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:55:43.413627 (+ 329us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:55:43.413895 (+ 268us) server_negotiation.cc:1035] Waiting for connection context
0809 22:55:43.414056 (+ 161us) server_negotiation.cc:299] Negotiation successful
0809 22:55:43.414267 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260809 22:55:43.415900 4855 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c64aba6229c14a26ac312e768826845f: Bootstrap starting.
I20260809 22:55:43.419193 15814 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 22:55:43.419638 4855 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c64aba6229c14a26ac312e768826845f: Neither blocks nor log segments found. Creating new log.
I20260809 22:55:43.420830 4855 log.cc:824] T 00000000000000000000000000000000 P c64aba6229c14a26ac312e768826845f: Log is configured to *not* fsync() on all Append() calls
I20260809 22:55:43.423925 4855 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c64aba6229c14a26ac312e768826845f: No bootstrap required, opened a new log
I20260809 22:55:43.434845 4855 raft_consensus.cc:348] T 00000000000000000000000000000000 P c64aba6229c14a26ac312e768826845f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0277478a7e37478c9e6d0c686f9236d6" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 40961 } } peers { permanent_uuid: "9bc5f2accd184ca987d8d463e8544c61" member_type: VOTER last_known_addr { host: "127.15.113.189" port: 38369 } } peers { permanent_uuid: "c64aba6229c14a26ac312e768826845f" member_type: VOTER last_known_addr { host: "127.15.113.188" port: 39007 } }
I20260809 22:55:43.435294 4855 raft_consensus.cc:374] T 00000000000000000000000000000000 P c64aba6229c14a26ac312e768826845f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:55:43.435449 4855 raft_consensus.cc:729] T 00000000000000000000000000000000 P c64aba6229c14a26ac312e768826845f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c64aba6229c14a26ac312e768826845f, State: Initialized, Role: FOLLOWER
I20260809 22:55:43.436255 4855 consensus_queue.cc:260] T 00000000000000000000000000000000 P c64aba6229c14a26ac312e768826845f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0277478a7e37478c9e6d0c686f9236d6" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 40961 } } peers { permanent_uuid: "9bc5f2accd184ca987d8d463e8544c61" member_type: VOTER last_known_addr { host: "127.15.113.189" port: 38369 } } peers { permanent_uuid: "c64aba6229c14a26ac312e768826845f" member_type: VOTER last_known_addr { host: "127.15.113.188" port: 39007 } }
I20260809 22:55:43.444535 4761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:55:43.424715 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40894 (local address 127.15.113.189:38369)
0809 22:55:43.425065 (+ 350us) server_negotiation.cc:206] Beginning negotiation
0809 22:55:43.425071 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:55:43.426438 (+ 1367us) server_negotiation.cc:407] Connection header received
0809 22:55:43.426570 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:55:43.426574 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:55:43.426652 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:55:43.426741 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:55:43.428970 (+ 2229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:55:43.430015 (+ 1045us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:55:43.432400 (+ 2385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:55:43.433018 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:55:43.433110 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:55:43.437754 (+ 4644us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:55:43.437760 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:55:43.437765 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:55:43.437801 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:55:43.441233 (+ 3432us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:55:43.442081 (+ 848us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:55:43.442086 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:55:43.442089 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:55:43.442260 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:55:43.442784 (+ 524us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:55:43.442790 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:55:43.442793 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:55:43.443145 (+ 352us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:55:43.443369 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0809 22:55:43.443975 (+ 606us) server_negotiation.cc:299] Negotiation successful
0809 22:55:43.444187 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":211}
I20260809 22:55:43.445801 4653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:55:43.424670 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56166 (local address 127.15.113.190:40961)
0809 22:55:43.424838 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0809 22:55:43.424844 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:55:43.425623 (+ 779us) server_negotiation.cc:407] Connection header received
0809 22:55:43.425771 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:55:43.425776 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:55:43.425853 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:55:43.425943 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:55:43.428932 (+ 2989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:55:43.430085 (+ 1153us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:55:43.432732 (+ 2647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:55:43.433485 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:55:43.433613 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:55:43.437685 (+ 4072us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:55:43.437693 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:55:43.437699 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:55:43.437750 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:55:43.441752 (+ 4002us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:55:43.442827 (+ 1075us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:55:43.442834 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:55:43.442837 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:55:43.443008 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:55:43.443501 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:55:43.443507 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:55:43.443510 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:55:43.443875 (+ 365us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:55:43.444101 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0809 22:55:43.445197 (+ 1096us) server_negotiation.cc:299] Negotiation successful
0809 22:55:43.445448 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260809 22:55:43.450160 4855 sys_catalog.cc:564] T 00000000000000000000000000000000 P c64aba6229c14a26ac312e768826845f [sys.catalog]: configured and running, proceeding with master startup.
I20260809 22:55:43.450420 4859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:55:43.426762 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53112 (local address 127.15.113.188:39007)
0809 22:55:43.426943 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0809 22:55:43.426949 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:55:43.434801 (+ 7852us) server_negotiation.cc:407] Connection header received
0809 22:55:43.434937 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:55:43.434943 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:55:43.435018 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:55:43.435123 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:55:43.437261 (+ 2138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:55:43.438291 (+ 1030us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:55:43.440129 (+ 1838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:55:43.440783 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:55:43.440876 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:55:43.444284 (+ 3408us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:55:43.444291 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:55:43.444296 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:55:43.444333 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:55:43.447339 (+ 3006us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:55:43.448169 (+ 830us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:55:43.448174 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:55:43.448177 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:55:43.448354 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:55:43.448812 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:55:43.448816 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:55:43.448820 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:55:43.449339 (+ 519us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:55:43.449561 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0809 22:55:43.449848 (+ 287us) server_negotiation.cc:299] Negotiation successful
0809 22:55:43.450076 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260809 22:55:43.451614 4875 sys_catalog.cc:455] T 00000000000000000000000000000000 P c64aba6229c14a26ac312e768826845f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0277478a7e37478c9e6d0c686f9236d6" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 40961 } } peers { permanent_uuid: "9bc5f2accd184ca987d8d463e8544c61" member_type: VOTER last_known_addr { host: "127.15.113.189" port: 38369 } } peers { permanent_uuid: "c64aba6229c14a26ac312e768826845f" member_type: VOTER last_known_addr { host: "127.15.113.188" port: 39007 } } }
I20260809 22:55:43.451920 4875 sys_catalog.cc:458] T 00000000000000000000000000000000 P c64aba6229c14a26ac312e768826845f [sys.catalog]: This master's current role is: FOLLOWER
W20260809 22:55:43.464540 4889 catalog_manager.cc:1558] T 00000000000000000000000000000000 P c64aba6229c14a26ac312e768826845f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 22:55:43.464727 4889 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 22:55:43.461668 4764 master_runner.cc:186] Error getting master registration for 127.15.113.190:40961: OK, instance_id { permanent_uuid: "0277478a7e37478c9e6d0c686f9236d6" instance_seqno: 1786316142361750 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 22:55:43.469821 4764 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260809 22:55:43.490497 4649 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.113.188" port: 39007 } attempt: 4
Aug 09 22:55:43 dist-test-slave-ncp9 krb5kdc[4530](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316142, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.113.190@KRBTEST.COM for kudu/127.15.113.188@KRBTEST.COM
I20260809 22:55:43.504017 4652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:55:43.491098 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.188:39007 (local address 127.0.0.1:53122)
0809 22:55:43.491293 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0809 22:55:43.491313 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 22:55:43.491432 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:55:43.491767 (+ 335us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:55:43.491772 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:55:43.492175 (+ 403us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:55:43.492342 (+ 167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:55:43.492351 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:55:43.493455 (+ 1104us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:55:43.493460 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:55:43.494389 (+ 929us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:55:43.494400 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:55:43.495115 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:55:43.495121 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:55:43.495296 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:55:43.495903 (+ 607us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:55:43.495921 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:55:43.498098 (+ 2177us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:55:43.501117 (+ 3019us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:55:43.501123 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:55:43.501127 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:55:43.501531 (+ 404us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:55:43.501994 (+ 463us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:55:43.501998 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:55:43.502002 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:55:43.502177 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:55:43.503078 (+ 901us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:55:43.503083 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:55:43.503228 (+ 145us) client_negotiation.cc:769] Sending connection context
0809 22:55:43.503358 (+ 130us) client_negotiation.cc:240] Negotiation successful
0809 22:55:43.503625 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":64}
I20260809 22:55:43.504068 4859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:55:43.491150 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53122 (local address 127.15.113.188:39007)
0809 22:55:43.491315 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0809 22:55:43.491320 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:55:43.491387 (+ 67us) server_negotiation.cc:407] Connection header received
0809 22:55:43.491546 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:55:43.491551 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:55:43.491636 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:55:43.491714 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:55:43.492489 (+ 775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:55:43.493329 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:55:43.494533 (+ 1204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:55:43.494981 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:55:43.495091 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:55:43.498275 (+ 3184us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:55:43.498280 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:55:43.498283 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:55:43.498313 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:55:43.501002 (+ 2689us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:55:43.501677 (+ 675us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:55:43.501683 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:55:43.501686 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:55:43.501854 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:55:43.502328 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:55:43.502333 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:55:43.502336 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:55:43.502948 (+ 612us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:55:43.503087 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0809 22:55:43.503480 (+ 393us) server_negotiation.cc:299] Negotiation successful
0809 22:55:43.503697 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260809 22:55:43.510226 4649 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0277478a7e37478c9e6d0c686f9236d6: Bootstrap starting.
I20260809 22:55:43.514508 4649 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0277478a7e37478c9e6d0c686f9236d6: Neither blocks nor log segments found. Creating new log.
I20260809 22:55:43.517189 4649 log.cc:824] T 00000000000000000000000000000000 P 0277478a7e37478c9e6d0c686f9236d6: Log is configured to *not* fsync() on all Append() calls
I20260809 22:55:43.520522 4649 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0277478a7e37478c9e6d0c686f9236d6: No bootstrap required, opened a new log
I20260809 22:55:43.531975 4649 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0277478a7e37478c9e6d0c686f9236d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0277478a7e37478c9e6d0c686f9236d6" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 40961 } } peers { permanent_uuid: "9bc5f2accd184ca987d8d463e8544c61" member_type: VOTER last_known_addr { host: "127.15.113.189" port: 38369 } } peers { permanent_uuid: "c64aba6229c14a26ac312e768826845f" member_type: VOTER last_known_addr { host: "127.15.113.188" port: 39007 } }
I20260809 22:55:43.532380 4649 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0277478a7e37478c9e6d0c686f9236d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:55:43.532510 4649 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0277478a7e37478c9e6d0c686f9236d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0277478a7e37478c9e6d0c686f9236d6, State: Initialized, Role: FOLLOWER
I20260809 22:55:43.533174 4649 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0277478a7e37478c9e6d0c686f9236d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0277478a7e37478c9e6d0c686f9236d6" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 40961 } } peers { permanent_uuid: "9bc5f2accd184ca987d8d463e8544c61" member_type: VOTER last_known_addr { host: "127.15.113.189" port: 38369 } } peers { permanent_uuid: "c64aba6229c14a26ac312e768826845f" member_type: VOTER last_known_addr { host: "127.15.113.188" port: 39007 } }
I20260809 22:55:43.534307 4894 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0277478a7e37478c9e6d0c686f9236d6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0277478a7e37478c9e6d0c686f9236d6" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 40961 } } peers { permanent_uuid: "9bc5f2accd184ca987d8d463e8544c61" member_type: VOTER last_known_addr { host: "127.15.113.189" port: 38369 } } peers { permanent_uuid: "c64aba6229c14a26ac312e768826845f" member_type: VOTER last_known_addr { host: "127.15.113.188" port: 39007 } } }
I20260809 22:55:43.534876 4894 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0277478a7e37478c9e6d0c686f9236d6 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 22:55:43.535548 4649 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0277478a7e37478c9e6d0c686f9236d6 [sys.catalog]: configured and running, proceeding with master startup.
W20260809 22:55:43.545441 4909 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 0277478a7e37478c9e6d0c686f9236d6: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 22:55:43.545573 4909 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 22:55:43.544929 4548 master_runner.cc:186] Error getting master registration for 127.15.113.189:38369: OK, instance_id { permanent_uuid: "9bc5f2accd184ca987d8d463e8544c61" instance_seqno: 1786316142847140 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 22:55:43.556784 4548 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260809 22:55:43.564970 4755 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.113.188" port: 39007 } attempt: 5
Aug 09 22:55:43 dist-test-slave-ncp9 krb5kdc[4530](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316142, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.113.189@KRBTEST.COM for kudu/127.15.113.188@KRBTEST.COM
W20260809 22:55:43.583000 4796 tablet.cc:2369] T 00000000000000000000000000000000 P c64aba6229c14a26ac312e768826845f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 22:55:43.584930 4759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:55:43.565619 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.188:39007 (local address 127.0.0.1:53138)
0809 22:55:43.565901 (+ 282us) negotiation.cc:107] Waiting for socket to connect
0809 22:55:43.565922 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0809 22:55:43.566042 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:55:43.566614 (+ 572us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:55:43.566619 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:55:43.567157 (+ 538us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:55:43.567375 (+ 218us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:55:43.567388 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:55:43.570181 (+ 2793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:55:43.570187 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:55:43.571593 (+ 1406us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:55:43.571609 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:55:43.572783 (+ 1174us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:55:43.572789 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:55:43.573017 (+ 228us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:55:43.573652 (+ 635us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:55:43.573677 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:55:43.576430 (+ 2753us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:55:43.580845 (+ 4415us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:55:43.580850 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:55:43.580855 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:55:43.581364 (+ 509us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:55:43.581812 (+ 448us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:55:43.581817 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:55:43.581820 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:55:43.582007 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:55:43.583924 (+ 1917us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:55:43.583930 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:55:43.584093 (+ 163us) client_negotiation.cc:769] Sending connection context
0809 22:55:43.584241 (+ 148us) client_negotiation.cc:240] Negotiation successful
0809 22:55:43.584521 (+ 280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":158}
I20260809 22:55:43.586079 4859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:55:43.565738 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53138 (local address 127.15.113.188:39007)
0809 22:55:43.565915 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0809 22:55:43.565922 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:55:43.566153 (+ 231us) server_negotiation.cc:407] Connection header received
0809 22:55:43.566223 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:55:43.566229 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:55:43.566309 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:55:43.566403 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:55:43.567551 (+ 1148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:55:43.568716 (+ 1165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:55:43.571776 (+ 3060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:55:43.572443 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:55:43.572564 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:55:43.576658 (+ 4094us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:55:43.576665 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:55:43.576671 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:55:43.576708 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:55:43.580693 (+ 3985us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:55:43.581499 (+ 806us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:55:43.581505 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:55:43.581508 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:55:43.581680 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:55:43.582134 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:55:43.582138 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:55:43.582141 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:55:43.582496 (+ 355us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:55:43.582671 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0809 22:55:43.585429 (+ 2758us) server_negotiation.cc:299] Negotiation successful
0809 22:55:43.585700 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260809 22:55:43.593962 4755 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9bc5f2accd184ca987d8d463e8544c61: Bootstrap starting.
I20260809 22:55:43.598241 4755 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9bc5f2accd184ca987d8d463e8544c61: Neither blocks nor log segments found. Creating new log.
I20260809 22:55:43.599552 4755 log.cc:824] T 00000000000000000000000000000000 P 9bc5f2accd184ca987d8d463e8544c61: Log is configured to *not* fsync() on all Append() calls
W20260809 22:55:43.601747 4580 tablet.cc:2369] T 00000000000000000000000000000000 P 0277478a7e37478c9e6d0c686f9236d6: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 22:55:43.602888 4755 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9bc5f2accd184ca987d8d463e8544c61: No bootstrap required, opened a new log
I20260809 22:55:43.615963 4755 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9bc5f2accd184ca987d8d463e8544c61 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0277478a7e37478c9e6d0c686f9236d6" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 40961 } } peers { permanent_uuid: "9bc5f2accd184ca987d8d463e8544c61" member_type: VOTER last_known_addr { host: "127.15.113.189" port: 38369 } } peers { permanent_uuid: "c64aba6229c14a26ac312e768826845f" member_type: VOTER last_known_addr { host: "127.15.113.188" port: 39007 } }
I20260809 22:55:43.616370 4755 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9bc5f2accd184ca987d8d463e8544c61 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:55:43.616487 4755 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9bc5f2accd184ca987d8d463e8544c61 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9bc5f2accd184ca987d8d463e8544c61, State: Initialized, Role: FOLLOWER
I20260809 22:55:43.617169 4755 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9bc5f2accd184ca987d8d463e8544c61 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0277478a7e37478c9e6d0c686f9236d6" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 40961 } } peers { permanent_uuid: "9bc5f2accd184ca987d8d463e8544c61" member_type: VOTER last_known_addr { host: "127.15.113.189" port: 38369 } } peers { permanent_uuid: "c64aba6229c14a26ac312e768826845f" member_type: VOTER last_known_addr { host: "127.15.113.188" port: 39007 } }
I20260809 22:55:43.618173 4915 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9bc5f2accd184ca987d8d463e8544c61 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0277478a7e37478c9e6d0c686f9236d6" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 40961 } } peers { permanent_uuid: "9bc5f2accd184ca987d8d463e8544c61" member_type: VOTER last_known_addr { host: "127.15.113.189" port: 38369 } } peers { permanent_uuid: "c64aba6229c14a26ac312e768826845f" member_type: VOTER last_known_addr { host: "127.15.113.188" port: 39007 } } }
I20260809 22:55:43.618618 4915 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9bc5f2accd184ca987d8d463e8544c61 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 22:55:43.619385 4755 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9bc5f2accd184ca987d8d463e8544c61 [sys.catalog]: configured and running, proceeding with master startup.
W20260809 22:55:43.641722 4931 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 9bc5f2accd184ca987d8d463e8544c61: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 22:55:43.641975 4931 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 22:55:43.647686 4658 master_runner.cc:263] No leader master found from master 9bc5f2accd184ca987d8d463e8544c61
I20260809 22:55:43.647842 4658 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260809 22:55:43.686739 4875 raft_consensus.cc:482] T 00000000000000000000000000000000 P c64aba6229c14a26ac312e768826845f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 22:55:43.687074 4875 raft_consensus.cc:504] T 00000000000000000000000000000000 P c64aba6229c14a26ac312e768826845f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0277478a7e37478c9e6d0c686f9236d6" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 40961 } } peers { permanent_uuid: "9bc5f2accd184ca987d8d463e8544c61" member_type: VOTER last_known_addr { host: "127.15.113.189" port: 38369 } } peers { permanent_uuid: "c64aba6229c14a26ac312e768826845f" member_type: VOTER last_known_addr { host: "127.15.113.188" port: 39007 } }
I20260809 22:55:43.688817 4875 leader_election.cc:288] T 00000000000000000000000000000000 P c64aba6229c14a26ac312e768826845f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0277478a7e37478c9e6d0c686f9236d6 (127.15.113.190:40961), 9bc5f2accd184ca987d8d463e8544c61 (127.15.113.189:38369)
I20260809 22:55:43.690300 4718 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "c64aba6229c14a26ac312e768826845f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9bc5f2accd184ca987d8d463e8544c61" is_pre_election: true
I20260809 22:55:43.690891 4718 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 9bc5f2accd184ca987d8d463e8544c61 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c64aba6229c14a26ac312e768826845f in term 0.
I20260809 22:55:43.691532 4788 leader_election.cc:302] T 00000000000000000000000000000000 P c64aba6229c14a26ac312e768826845f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9bc5f2accd184ca987d8d463e8544c61, c64aba6229c14a26ac312e768826845f; no voters:
I20260809 22:55:43.692279 4619 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "c64aba6229c14a26ac312e768826845f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0277478a7e37478c9e6d0c686f9236d6" is_pre_election: true
I20260809 22:55:43.696278 4619 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 0277478a7e37478c9e6d0c686f9236d6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c64aba6229c14a26ac312e768826845f in term 0.
I20260809 22:55:43.698904 4875 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c64aba6229c14a26ac312e768826845f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 22:55:43.699059 4875 raft_consensus.cc:482] T 00000000000000000000000000000000 P c64aba6229c14a26ac312e768826845f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 22:55:43.699177 4875 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c64aba6229c14a26ac312e768826845f [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:55:43.702697 4875 raft_consensus.cc:504] T 00000000000000000000000000000000 P c64aba6229c14a26ac312e768826845f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0277478a7e37478c9e6d0c686f9236d6" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 40961 } } peers { permanent_uuid: "9bc5f2accd184ca987d8d463e8544c61" member_type: VOTER last_known_addr { host: "127.15.113.189" port: 38369 } } peers { permanent_uuid: "c64aba6229c14a26ac312e768826845f" member_type: VOTER last_known_addr { host: "127.15.113.188" port: 39007 } }
I20260809 22:55:43.703790 4619 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "c64aba6229c14a26ac312e768826845f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0277478a7e37478c9e6d0c686f9236d6"
I20260809 22:55:43.704018 4619 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0277478a7e37478c9e6d0c686f9236d6 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:55:43.707732 4619 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 0277478a7e37478c9e6d0c686f9236d6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c64aba6229c14a26ac312e768826845f in term 1.
I20260809 22:55:43.708251 4791 leader_election.cc:302] T 00000000000000000000000000000000 P c64aba6229c14a26ac312e768826845f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0277478a7e37478c9e6d0c686f9236d6, c64aba6229c14a26ac312e768826845f; no voters:
I20260809 22:55:43.711876 4718 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "c64aba6229c14a26ac312e768826845f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9bc5f2accd184ca987d8d463e8544c61"
I20260809 22:55:43.712105 4718 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9bc5f2accd184ca987d8d463e8544c61 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:55:43.715483 4718 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 9bc5f2accd184ca987d8d463e8544c61 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c64aba6229c14a26ac312e768826845f in term 1.
I20260809 22:55:43.716161 4875 leader_election.cc:288] T 00000000000000000000000000000000 P c64aba6229c14a26ac312e768826845f [CANDIDATE]: Term 1 election: Requested vote from peers 0277478a7e37478c9e6d0c686f9236d6 (127.15.113.190:40961), 9bc5f2accd184ca987d8d463e8544c61 (127.15.113.189:38369)
I20260809 22:55:43.716357 4875 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c64aba6229c14a26ac312e768826845f [term 1 FOLLOWER]: Leader election won for term 1
I20260809 22:55:43.716493 4875 raft_consensus.cc:686] T 00000000000000000000000000000000 P c64aba6229c14a26ac312e768826845f [term 1 LEADER]: Becoming Leader. State: Replica: c64aba6229c14a26ac312e768826845f, State: Running, Role: LEADER
I20260809 22:55:43.717046 4875 consensus_queue.cc:237] T 00000000000000000000000000000000 P c64aba6229c14a26ac312e768826845f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0277478a7e37478c9e6d0c686f9236d6" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 40961 } } peers { permanent_uuid: "9bc5f2accd184ca987d8d463e8544c61" member_type: VOTER last_known_addr { host: "127.15.113.189" port: 38369 } } peers { permanent_uuid: "c64aba6229c14a26ac312e768826845f" member_type: VOTER last_known_addr { host: "127.15.113.188" port: 39007 } }
I20260809 22:55:43.735258 4933 sys_catalog.cc:455] T 00000000000000000000000000000000 P c64aba6229c14a26ac312e768826845f [sys.catalog]: SysCatalogTable state changed. Reason: New leader c64aba6229c14a26ac312e768826845f. Latest consensus state: current_term: 1 leader_uuid: "c64aba6229c14a26ac312e768826845f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0277478a7e37478c9e6d0c686f9236d6" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 40961 } } peers { permanent_uuid: "9bc5f2accd184ca987d8d463e8544c61" member_type: VOTER last_known_addr { host: "127.15.113.189" port: 38369 } } peers { permanent_uuid: "c64aba6229c14a26ac312e768826845f" member_type: VOTER last_known_addr { host: "127.15.113.188" port: 39007 } } }
I20260809 22:55:43.735697 4933 sys_catalog.cc:458] T 00000000000000000000000000000000 P c64aba6229c14a26ac312e768826845f [sys.catalog]: This master's current role is: LEADER
I20260809 22:55:43.738648 4937 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 22:55:43.740834 4937 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 22:55:43.749591 4718 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 9bc5f2accd184ca987d8d463e8544c61 [term 1 FOLLOWER]: Refusing update from remote peer c64aba6229c14a26ac312e768826845f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 22:55:43.750780 4933 consensus_queue.cc:1035] T 00000000000000000000000000000000 P c64aba6229c14a26ac312e768826845f [LEADER]: Connected to new peer: Peer: permanent_uuid: "9bc5f2accd184ca987d8d463e8544c61" member_type: VOTER last_known_addr { host: "127.15.113.189" port: 38369 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 22:55:43.764569 4619 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 0277478a7e37478c9e6d0c686f9236d6 [term 1 FOLLOWER]: Refusing update from remote peer c64aba6229c14a26ac312e768826845f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 22:55:43.765472 4933 consensus_queue.cc:1035] T 00000000000000000000000000000000 P c64aba6229c14a26ac312e768826845f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0277478a7e37478c9e6d0c686f9236d6" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 40961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 22:55:43.788478 4894 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0277478a7e37478c9e6d0c686f9236d6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c64aba6229c14a26ac312e768826845f. Latest consensus state: current_term: 1 leader_uuid: "c64aba6229c14a26ac312e768826845f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0277478a7e37478c9e6d0c686f9236d6" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 40961 } } peers { permanent_uuid: "9bc5f2accd184ca987d8d463e8544c61" member_type: VOTER last_known_addr { host: "127.15.113.189" port: 38369 } } peers { permanent_uuid: "c64aba6229c14a26ac312e768826845f" member_type: VOTER last_known_addr { host: "127.15.113.188" port: 39007 } } }
I20260809 22:55:43.788815 4894 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0277478a7e37478c9e6d0c686f9236d6 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 22:55:43.792598 4894 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0277478a7e37478c9e6d0c686f9236d6 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c64aba6229c14a26ac312e768826845f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0277478a7e37478c9e6d0c686f9236d6" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 40961 } } peers { permanent_uuid: "9bc5f2accd184ca987d8d463e8544c61" member_type: VOTER last_known_addr { host: "127.15.113.189" port: 38369 } } peers { permanent_uuid: "c64aba6229c14a26ac312e768826845f" member_type: VOTER last_known_addr { host: "127.15.113.188" port: 39007 } } }
I20260809 22:55:43.792837 4894 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0277478a7e37478c9e6d0c686f9236d6 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 22:55:43.798130 4938 mvcc.cc:205] Tried to move back new op lower bound from 7316750924787937280 to 7316750924702384128. Current Snapshot: MvccSnapshot[applied={T|T < 7316750924787937280}]
I20260809 22:55:43.807120 4915 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9bc5f2accd184ca987d8d463e8544c61 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c64aba6229c14a26ac312e768826845f. Latest consensus state: current_term: 1 leader_uuid: "c64aba6229c14a26ac312e768826845f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0277478a7e37478c9e6d0c686f9236d6" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 40961 } } peers { permanent_uuid: "9bc5f2accd184ca987d8d463e8544c61" member_type: VOTER last_known_addr { host: "127.15.113.189" port: 38369 } } peers { permanent_uuid: "c64aba6229c14a26ac312e768826845f" member_type: VOTER last_known_addr { host: "127.15.113.188" port: 39007 } } }
I20260809 22:55:43.807442 4915 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9bc5f2accd184ca987d8d463e8544c61 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 22:55:43.807307 4875 sys_catalog.cc:455] T 00000000000000000000000000000000 P c64aba6229c14a26ac312e768826845f [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c64aba6229c14a26ac312e768826845f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0277478a7e37478c9e6d0c686f9236d6" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 40961 } } peers { permanent_uuid: "9bc5f2accd184ca987d8d463e8544c61" member_type: VOTER last_known_addr { host: "127.15.113.189" port: 38369 } } peers { permanent_uuid: "c64aba6229c14a26ac312e768826845f" member_type: VOTER last_known_addr { host: "127.15.113.188" port: 39007 } } }
I20260809 22:55:43.807595 4875 sys_catalog.cc:458] T 00000000000000000000000000000000 P c64aba6229c14a26ac312e768826845f [sys.catalog]: This master's current role is: LEADER
I20260809 22:55:43.816368 4915 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9bc5f2accd184ca987d8d463e8544c61 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c64aba6229c14a26ac312e768826845f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0277478a7e37478c9e6d0c686f9236d6" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 40961 } } peers { permanent_uuid: "9bc5f2accd184ca987d8d463e8544c61" member_type: VOTER last_known_addr { host: "127.15.113.189" port: 38369 } } peers { permanent_uuid: "c64aba6229c14a26ac312e768826845f" member_type: VOTER last_known_addr { host: "127.15.113.188" port: 39007 } } }
I20260809 22:55:43.816675 4915 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9bc5f2accd184ca987d8d463e8544c61 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 22:55:43.821410 4933 sys_catalog.cc:455] T 00000000000000000000000000000000 P c64aba6229c14a26ac312e768826845f [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c64aba6229c14a26ac312e768826845f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0277478a7e37478c9e6d0c686f9236d6" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 40961 } } peers { permanent_uuid: "9bc5f2accd184ca987d8d463e8544c61" member_type: VOTER last_known_addr { host: "127.15.113.189" port: 38369 } } peers { permanent_uuid: "c64aba6229c14a26ac312e768826845f" member_type: VOTER last_known_addr { host: "127.15.113.188" port: 39007 } } }
I20260809 22:55:43.822714 4933 sys_catalog.cc:458] T 00000000000000000000000000000000 P c64aba6229c14a26ac312e768826845f [sys.catalog]: This master's current role is: LEADER
I20260809 22:55:43.829556 4937 catalog_manager.cc:1347] Generated new cluster ID: b0ade9a7619c4b47b1e590cc25e20a11
I20260809 22:55:43.829679 4937 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 22:55:43.832863 4942 mvcc.cc:205] Tried to move back new op lower bound from 7316750924787937280 to 7316750924702384128. Current Snapshot: MvccSnapshot[applied={T|T < 7316750924787937280 or (T in {7316750924787937280})}]
I20260809 22:55:43.834251 4939 mvcc.cc:205] Tried to move back new op lower bound from 7316750924787937280 to 7316750924702384128. Current Snapshot: MvccSnapshot[applied={T|T < 7316750924787937280 or (T in {7316750924787937280})}]
I20260809 22:55:43.847764 4937 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 22:55:43.849735 4937 catalog_manager.cc:1504] Loading token signing keys...
I20260809 22:55:43.870803 4937 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c64aba6229c14a26ac312e768826845f: Generated new TSK 0
I20260809 22:55:43.871820 4937 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 22:55:44.547641 4909 catalog_manager.cc:1259] Loaded cluster ID: b0ade9a7619c4b47b1e590cc25e20a11
I20260809 22:55:44.547766 4909 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 0277478a7e37478c9e6d0c686f9236d6: loading cluster ID for follower catalog manager: success
I20260809 22:55:44.550319 4909 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 0277478a7e37478c9e6d0c686f9236d6: acquiring CA information for follower catalog manager: success
I20260809 22:55:44.552059 4909 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 0277478a7e37478c9e6d0c686f9236d6: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 22:55:44.643967 4931 catalog_manager.cc:1259] Loaded cluster ID: b0ade9a7619c4b47b1e590cc25e20a11
I20260809 22:55:44.644081 4931 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 9bc5f2accd184ca987d8d463e8544c61: loading cluster ID for follower catalog manager: success
I20260809 22:55:44.646564 4931 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 9bc5f2accd184ca987d8d463e8544c61: acquiring CA information for follower catalog manager: success
I20260809 22:55:44.648343 4931 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 9bc5f2accd184ca987d8d463e8544c61: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 22:55:45.012239 15814 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskItYPJR/build/asan/bin/kudu pid 4548
I20260809 22:55:45.013918 4643 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:55:45.121183 15814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskItYPJR/build/asan/bin/kudu with pid 4548
I20260809 22:55:45.148844 15814 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskItYPJR/build/asan/bin/kudu pid 4658
I20260809 22:55:45.149951 4743 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:55:45.253696 15814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskItYPJR/build/asan/bin/kudu with pid 4658
I20260809 22:55:45.279740 15814 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskItYPJR/build/asan/bin/kudu pid 4764
I20260809 22:55:45.280745 4847 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:55:45.388720 15814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskItYPJR/build/asan/bin/kudu with pid 4764
2026-08-09T22:55:45Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (4910 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786315935617919-15814-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 22:55:45 dist-test-slave-ncp9 krb5kdc[4965](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 22:55:45 dist-test-slave-ncp9 krb5kdc[4965](info): set up 2 sockets
Aug 09 22:55:45 dist-test-slave-ncp9 krb5kdc[4965](info): commencing operation
krb5kdc: starting...
W20260809 22:55:46.594810 15814 mini_kdc.cc:121] Time spent starting KDC: real 1.161s user 0.000s sys 0.035s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786315935617919-15814-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786315935617919-15814-0/krb5kdc/test-user.keytab.
Aug 09 22:55:46 dist-test-slave-ncp9 krb5kdc[4965](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316146, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T22:55:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T22:55:46Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.113.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.113.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.113.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.190.keytab.
Entry for principal kudu/127.15.113.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.190.keytab.
I20260809 22:55:46.850579 15814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskItYPJR/build/asan/bin/kudu
/tmp/dist-test-taskItYPJR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786315935617919-15814-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786315935617919-15814-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786315935617919-15814-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786315935617919-15814-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.113.190:45621
--webserver_interface=127.15.113.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.190.keytab
--principal=kudu/127.15.113.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.113.148:45931
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.113.190:45621
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786315935617919-15814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786315935617919-15814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786315935617919-15814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:55:47.177811 4983 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:55:47.178088 4983 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:55:47.178220 4983 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:55:47.185254 4983 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 22:55:47.185357 4983 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:55:47.185427 4983 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:55:47.185511 4983 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 22:55:47.185570 4983 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 22:55:47.195389 4983 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.113.148:45931
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786315935617919-15814-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786315935617919-15814-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.113.190:45621
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.190.keytab
--principal=kudu/127.15.113.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.113.190:45621
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786315935617919-15814-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.113.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786315935617919-15814-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:55:47.196657 4983 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:55:47.197988 4983 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:55:47.209164 4990 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:55:47.209254 4994 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:55:47.209281 4997 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:55:47.210440 4983 server_base.cc:1034] running on GCE node
I20260809 22:55:47.211100 4983 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:55:47.212337 4983 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:55:47.213558 4983 hybrid_clock.cc:648] HybridClock initialized: now 1786316147213515 us; error 30 us; skew 500 ppm
I20260809 22:55:47.213876 4983 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:55:47 dist-test-slave-ncp9 krb5kdc[4965](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316147, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.113.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:55:47.216826 4983 init.cc:377] Logged in from keytab as kudu/127.15.113.190@KRBTEST.COM (short username kudu)
I20260809 22:55:47.218278 4983 webserver.cc:460] Webserver started at http://127.15.113.190:34117/ using document root <none> and password file <none>
I20260809 22:55:47.218901 4983 fs_manager.cc:359] Metadata directory not provided
I20260809 22:55:47.219004 4983 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:55:47.219256 4983 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:55:47.222175 4983 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786315935617919-15814-0/minicluster-data/master-0/data/instance:
uuid: "920bdb4e43ee448ab535b14fedb356a4"
format_stamp: "Formatted at 2026-08-09 22:55:47 on dist-test-slave-ncp9"
server_key: "7e0034cddf580abdf97c79031c7db0e8"
server_key_iv: "f7e30aed120604924e8b98220bef3c29"
server_key_version: "encryptionkey@0"
I20260809 22:55:47.222885 4983 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786315935617919-15814-0/minicluster-data/master-0/wal/instance:
uuid: "920bdb4e43ee448ab535b14fedb356a4"
format_stamp: "Formatted at 2026-08-09 22:55:47 on dist-test-slave-ncp9"
server_key: "7e0034cddf580abdf97c79031c7db0e8"
server_key_iv: "f7e30aed120604924e8b98220bef3c29"
server_key_version: "encryptionkey@0"
I20260809 22:55:47.227730 4983 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 22:55:47.231065 5007 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:55:47.232314 4983 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 22:55:47.232472 4983 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786315935617919-15814-0/minicluster-data/master-0/data,/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786315935617919-15814-0/minicluster-data/master-0/wal
uuid: "920bdb4e43ee448ab535b14fedb356a4"
format_stamp: "Formatted at 2026-08-09 22:55:47 on dist-test-slave-ncp9"
server_key: "7e0034cddf580abdf97c79031c7db0e8"
server_key_iv: "f7e30aed120604924e8b98220bef3c29"
server_key_version: "encryptionkey@0"
I20260809 22:55:47.232666 4983 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786315935617919-15814-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786315935617919-15814-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786315935617919-15814-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:55:47.253815 4983 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:55:47.257470 4983 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:55:47.257728 4983 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:55:47.274763 4983 rpc_server.cc:307] RPC server started. Bound to: 127.15.113.190:45621
I20260809 22:55:47.274785 5088 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.113.190:45621 every 8 connection(s)
I20260809 22:55:47.275990 4983 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786315935617919-15814-0/minicluster-data/master-0/data/info.pb
I20260809 22:55:47.279520 5090 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:55:47.281901 15814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskItYPJR/build/asan/bin/kudu as pid 4983
I20260809 22:55:47.282135 15814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786315935617919-15814-0/minicluster-data/master-0/wal/instance
I20260809 22:55:47.282984 15814 external_mini_cluster.cc:1428] Setting key 542a1ee7f5722097d356532936579ac2
I20260809 22:55:47.291575 5090 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 920bdb4e43ee448ab535b14fedb356a4: Bootstrap starting.
I20260809 22:55:47.295221 5090 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 920bdb4e43ee448ab535b14fedb356a4: Neither blocks nor log segments found. Creating new log.
I20260809 22:55:47.296481 5090 log.cc:824] T 00000000000000000000000000000000 P 920bdb4e43ee448ab535b14fedb356a4: Log is configured to *not* fsync() on all Append() calls
Aug 09 22:55:47 dist-test-slave-ncp9 krb5kdc[4965](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316146, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.113.190@KRBTEST.COM
I20260809 22:55:47.299520 5090 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 920bdb4e43ee448ab535b14fedb356a4: No bootstrap required, opened a new log
I20260809 22:55:47.306129 5095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:55:47.284862 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45140 (local address 127.15.113.190:45621)
0809 22:55:47.285706 (+ 844us) server_negotiation.cc:206] Beginning negotiation
0809 22:55:47.285718 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 22:55:47.285759 (+ 41us) server_negotiation.cc:407] Connection header received
0809 22:55:47.287392 (+ 1633us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:55:47.287414 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:55:47.287780 (+ 366us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:55:47.288110 (+ 330us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:55:47.289857 (+ 1747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:55:47.291187 (+ 1330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:55:47.293101 (+ 1914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:55:47.293763 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:55:47.293898 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:55:47.299122 (+ 5224us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:55:47.299130 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:55:47.299143 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:55:47.299176 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:55:47.302228 (+ 3052us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:55:47.303032 (+ 804us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:55:47.303038 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:55:47.303047 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:55:47.303210 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:55:47.303575 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:55:47.303578 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:55:47.303580 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:55:47.304089 (+ 509us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:55:47.304318 (+ 229us) server_negotiation.cc:1035] Waiting for connection context
0809 22:55:47.304753 (+ 435us) server_negotiation.cc:299] Negotiation successful
0809 22:55:47.305146 (+ 393us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":643,"thread_start_us":360,"threads_started":1}
I20260809 22:55:47.307968 5090 raft_consensus.cc:348] T 00000000000000000000000000000000 P 920bdb4e43ee448ab535b14fedb356a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "920bdb4e43ee448ab535b14fedb356a4" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 45621 } }
I20260809 22:55:47.308315 5090 raft_consensus.cc:374] T 00000000000000000000000000000000 P 920bdb4e43ee448ab535b14fedb356a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:55:47.308411 5090 raft_consensus.cc:729] T 00000000000000000000000000000000 P 920bdb4e43ee448ab535b14fedb356a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 920bdb4e43ee448ab535b14fedb356a4, State: Initialized, Role: FOLLOWER
I20260809 22:55:47.309021 5090 consensus_queue.cc:260] T 00000000000000000000000000000000 P 920bdb4e43ee448ab535b14fedb356a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "920bdb4e43ee448ab535b14fedb356a4" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 45621 } }
I20260809 22:55:47.309262 5090 raft_consensus.cc:388] T 00000000000000000000000000000000 P 920bdb4e43ee448ab535b14fedb356a4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 22:55:47.309403 5090 raft_consensus.cc:482] T 00000000000000000000000000000000 P 920bdb4e43ee448ab535b14fedb356a4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 22:55:47.309563 5090 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 920bdb4e43ee448ab535b14fedb356a4 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:55:47.309651 15814 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 22:55:47.311589 5090 raft_consensus.cc:504] T 00000000000000000000000000000000 P 920bdb4e43ee448ab535b14fedb356a4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "920bdb4e43ee448ab535b14fedb356a4" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 45621 } }
I20260809 22:55:47.312093 5090 leader_election.cc:302] T 00000000000000000000000000000000 P 920bdb4e43ee448ab535b14fedb356a4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 920bdb4e43ee448ab535b14fedb356a4; no voters:
I20260809 22:55:47.312779 5090 leader_election.cc:288] T 00000000000000000000000000000000 P 920bdb4e43ee448ab535b14fedb356a4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 22:55:47.312861 5102 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 920bdb4e43ee448ab535b14fedb356a4 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 22:55:47.313555 5102 raft_consensus.cc:686] T 00000000000000000000000000000000 P 920bdb4e43ee448ab535b14fedb356a4 [term 1 LEADER]: Becoming Leader. State: Replica: 920bdb4e43ee448ab535b14fedb356a4, State: Running, Role: LEADER
I20260809 22:55:47.314055 5102 consensus_queue.cc:237] T 00000000000000000000000000000000 P 920bdb4e43ee448ab535b14fedb356a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "920bdb4e43ee448ab535b14fedb356a4" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 45621 } }
I20260809 22:55:47.314642 5090 sys_catalog.cc:564] T 00000000000000000000000000000000 P 920bdb4e43ee448ab535b14fedb356a4 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 22:55:47.316886 5103 sys_catalog.cc:455] T 00000000000000000000000000000000 P 920bdb4e43ee448ab535b14fedb356a4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "920bdb4e43ee448ab535b14fedb356a4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "920bdb4e43ee448ab535b14fedb356a4" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 45621 } } }
I20260809 22:55:47.317535 5103 sys_catalog.cc:458] T 00000000000000000000000000000000 P 920bdb4e43ee448ab535b14fedb356a4 [sys.catalog]: This master's current role is: LEADER
I20260809 22:55:47.317605 5105 sys_catalog.cc:455] T 00000000000000000000000000000000 P 920bdb4e43ee448ab535b14fedb356a4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 920bdb4e43ee448ab535b14fedb356a4. Latest consensus state: current_term: 1 leader_uuid: "920bdb4e43ee448ab535b14fedb356a4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "920bdb4e43ee448ab535b14fedb356a4" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 45621 } } }
I20260809 22:55:47.317902 5105 sys_catalog.cc:458] T 00000000000000000000000000000000 P 920bdb4e43ee448ab535b14fedb356a4 [sys.catalog]: This master's current role is: LEADER
I20260809 22:55:47.321725 5115 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 22:55:47.326577 5115 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 22:55:47.334630 5115 catalog_manager.cc:1347] Generated new cluster ID: bc97d7d7c6564d43aa4084ddfda49de1
I20260809 22:55:47.334733 5115 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 22:55:47.364817 5115 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 22:55:47.366034 5115 catalog_manager.cc:1504] Loading token signing keys...
I20260809 22:55:47.373205 5115 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 920bdb4e43ee448ab535b14fedb356a4: Generated new TSK 0
I20260809 22:55:47.373909 5115 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 22:55:47.420564 15814 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskItYPJR/build/asan/bin/kudu pid 4983
I20260809 22:55:47.421638 5078 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:55:47.520303 15814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskItYPJR/build/asan/bin/kudu with pid 4983
2026-08-09T22:55:47Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (2128 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315935617919-15814-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 22:55:47 dist-test-slave-ncp9 krb5kdc[5135](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 22:55:47 dist-test-slave-ncp9 krb5kdc[5135](info): set up 2 sockets
Aug 09 22:55:47 dist-test-slave-ncp9 krb5kdc[5135](info): commencing operation
krb5kdc: starting...
W20260809 22:55:48.756273 15814 mini_kdc.cc:121] Time spent starting KDC: real 1.194s user 0.004s sys 0.032s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315935617919-15814-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315935617919-15814-0/krb5kdc/test-user.keytab.
Aug 09 22:55:48 dist-test-slave-ncp9 krb5kdc[5135](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316148, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T22:55:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T22:55:48Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.113.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.113.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.113.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.190.keytab.
Entry for principal kudu/127.15.113.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.190.keytab.
I20260809 22:55:49.005227 15814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskItYPJR/build/asan/bin/kudu
/tmp/dist-test-taskItYPJR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315935617919-15814-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315935617919-15814-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315935617919-15814-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315935617919-15814-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.113.190:46137
--webserver_interface=127.15.113.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.190.keytab
--principal=kudu/127.15.113.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.113.148:34295
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.113.190:46137
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315935617919-15814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315935617919-15814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315935617919-15814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:55:49.337673 5157 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:55:49.337942 5157 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:55:49.338070 5157 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:55:49.345032 5157 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 22:55:49.345139 5157 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:55:49.345221 5157 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:55:49.345296 5157 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 22:55:49.345353 5157 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 22:55:49.354604 5157 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.113.148:34295
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315935617919-15814-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315935617919-15814-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.113.190:46137
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.190.keytab
--principal=kudu/127.15.113.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.113.190:46137
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315935617919-15814-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.113.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315935617919-15814-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:55:49.355816 5157 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:55:49.357102 5157 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:55:49.367966 5167 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:55:49.368021 5164 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:55:49.368000 5165 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:55:49.368732 5157 server_base.cc:1034] running on GCE node
I20260809 22:55:49.369494 5157 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:55:49.370709 5157 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:55:49.371907 5157 hybrid_clock.cc:648] HybridClock initialized: now 1786316149371856 us; error 40 us; skew 500 ppm
I20260809 22:55:49.372229 5157 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:55:49 dist-test-slave-ncp9 krb5kdc[5135](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316149, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.113.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:55:49.375129 5157 init.cc:377] Logged in from keytab as kudu/127.15.113.190@KRBTEST.COM (short username kudu)
I20260809 22:55:49.376435 5157 webserver.cc:460] Webserver started at http://127.15.113.190:45615/ using document root <none> and password file <none>
I20260809 22:55:49.377153 5157 fs_manager.cc:359] Metadata directory not provided
I20260809 22:55:49.377269 5157 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:55:49.377544 5157 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:55:49.380332 5157 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315935617919-15814-0/minicluster-data/master-0/data/instance:
uuid: "60c579fec9964c88b77f838d9c13df36"
format_stamp: "Formatted at 2026-08-09 22:55:49 on dist-test-slave-ncp9"
server_key: "b1fa6de1da0146b96208386a306420e6"
server_key_iv: "1740c3981367fca234f4799be5a4c375"
server_key_version: "encryptionkey@0"
I20260809 22:55:49.381096 5157 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315935617919-15814-0/minicluster-data/master-0/wal/instance:
uuid: "60c579fec9964c88b77f838d9c13df36"
format_stamp: "Formatted at 2026-08-09 22:55:49 on dist-test-slave-ncp9"
server_key: "b1fa6de1da0146b96208386a306420e6"
server_key_iv: "1740c3981367fca234f4799be5a4c375"
server_key_version: "encryptionkey@0"
I20260809 22:55:49.386698 5157 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260809 22:55:49.389931 5176 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:55:49.391160 5157 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.004s
I20260809 22:55:49.391319 5157 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315935617919-15814-0/minicluster-data/master-0/data,/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315935617919-15814-0/minicluster-data/master-0/wal
uuid: "60c579fec9964c88b77f838d9c13df36"
format_stamp: "Formatted at 2026-08-09 22:55:49 on dist-test-slave-ncp9"
server_key: "b1fa6de1da0146b96208386a306420e6"
server_key_iv: "1740c3981367fca234f4799be5a4c375"
server_key_version: "encryptionkey@0"
I20260809 22:55:49.391484 5157 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315935617919-15814-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315935617919-15814-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315935617919-15814-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:55:49.414084 5157 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:55:49.417613 5157 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:55:49.417855 5157 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:55:49.434767 5157 rpc_server.cc:307] RPC server started. Bound to: 127.15.113.190:46137
I20260809 22:55:49.434823 5250 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.113.190:46137 every 8 connection(s)
I20260809 22:55:49.435989 5157 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315935617919-15814-0/minicluster-data/master-0/data/info.pb
I20260809 22:55:49.438179 15814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskItYPJR/build/asan/bin/kudu as pid 5157
I20260809 22:55:49.438362 15814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315935617919-15814-0/minicluster-data/master-0/wal/instance
I20260809 22:55:49.439033 15814 external_mini_cluster.cc:1428] Setting key 9bd047cbf02b6c93482212401a4e0acc
I20260809 22:55:49.439539 5251 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:55:49.453871 5251 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 60c579fec9964c88b77f838d9c13df36: Bootstrap starting.
Aug 09 22:55:49 dist-test-slave-ncp9 krb5kdc[5135](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316148, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.113.190@KRBTEST.COM
I20260809 22:55:49.457288 5251 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 60c579fec9964c88b77f838d9c13df36: Neither blocks nor log segments found. Creating new log.
I20260809 22:55:49.458616 5251 log.cc:824] T 00000000000000000000000000000000 P 60c579fec9964c88b77f838d9c13df36: Log is configured to *not* fsync() on all Append() calls
I20260809 22:55:49.461654 5251 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 60c579fec9964c88b77f838d9c13df36: No bootstrap required, opened a new log
I20260809 22:55:49.464370 5256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:55:49.440821 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58064 (local address 127.15.113.190:46137)
0809 22:55:49.441577 (+ 756us) server_negotiation.cc:206] Beginning negotiation
0809 22:55:49.441588 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 22:55:49.441627 (+ 39us) server_negotiation.cc:407] Connection header received
0809 22:55:49.444603 (+ 2976us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:55:49.444633 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:55:49.445162 (+ 529us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:55:49.445583 (+ 421us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:55:49.447338 (+ 1755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:55:49.449333 (+ 1995us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:55:49.451272 (+ 1939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:55:49.452326 (+ 1054us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:55:49.452501 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:55:49.456668 (+ 4167us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:55:49.456676 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:55:49.456689 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:55:49.456723 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:55:49.460106 (+ 3383us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:55:49.460781 (+ 675us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:55:49.460790 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:55:49.460797 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:55:49.461023 (+ 226us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:55:49.461381 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:55:49.461386 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:55:49.461388 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:55:49.462056 (+ 668us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:55:49.462389 (+ 333us) server_negotiation.cc:1035] Waiting for connection context
0809 22:55:49.462687 (+ 298us) server_negotiation.cc:299] Negotiation successful
0809 22:55:49.463088 (+ 401us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":572,"thread_start_us":327,"threads_started":1}
I20260809 22:55:49.470253 5251 raft_consensus.cc:348] T 00000000000000000000000000000000 P 60c579fec9964c88b77f838d9c13df36 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60c579fec9964c88b77f838d9c13df36" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 46137 } }
I20260809 22:55:49.470572 5251 raft_consensus.cc:374] T 00000000000000000000000000000000 P 60c579fec9964c88b77f838d9c13df36 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:55:49.470664 5251 raft_consensus.cc:729] T 00000000000000000000000000000000 P 60c579fec9964c88b77f838d9c13df36 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60c579fec9964c88b77f838d9c13df36, State: Initialized, Role: FOLLOWER
I20260809 22:55:49.471240 5251 consensus_queue.cc:260] T 00000000000000000000000000000000 P 60c579fec9964c88b77f838d9c13df36 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60c579fec9964c88b77f838d9c13df36" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 46137 } }
I20260809 22:55:49.471470 5251 raft_consensus.cc:388] T 00000000000000000000000000000000 P 60c579fec9964c88b77f838d9c13df36 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 22:55:49.471571 5251 raft_consensus.cc:482] T 00000000000000000000000000000000 P 60c579fec9964c88b77f838d9c13df36 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 22:55:49.471700 5251 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 60c579fec9964c88b77f838d9c13df36 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:55:49.473716 5251 raft_consensus.cc:504] T 00000000000000000000000000000000 P 60c579fec9964c88b77f838d9c13df36 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60c579fec9964c88b77f838d9c13df36" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 46137 } }
I20260809 22:55:49.474145 5251 leader_election.cc:302] T 00000000000000000000000000000000 P 60c579fec9964c88b77f838d9c13df36 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 60c579fec9964c88b77f838d9c13df36; no voters:
I20260809 22:55:49.474615 5251 leader_election.cc:288] T 00000000000000000000000000000000 P 60c579fec9964c88b77f838d9c13df36 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 22:55:49.474747 5258 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 60c579fec9964c88b77f838d9c13df36 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 22:55:49.475363 5258 raft_consensus.cc:686] T 00000000000000000000000000000000 P 60c579fec9964c88b77f838d9c13df36 [term 1 LEADER]: Becoming Leader. State: Replica: 60c579fec9964c88b77f838d9c13df36, State: Running, Role: LEADER
I20260809 22:55:49.475911 5258 consensus_queue.cc:237] T 00000000000000000000000000000000 P 60c579fec9964c88b77f838d9c13df36 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60c579fec9964c88b77f838d9c13df36" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 46137 } }
I20260809 22:55:49.476438 5251 sys_catalog.cc:564] T 00000000000000000000000000000000 P 60c579fec9964c88b77f838d9c13df36 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 22:55:49.478403 5259 sys_catalog.cc:455] T 00000000000000000000000000000000 P 60c579fec9964c88b77f838d9c13df36 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "60c579fec9964c88b77f838d9c13df36" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60c579fec9964c88b77f838d9c13df36" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 46137 } } }
I20260809 22:55:49.478799 5259 sys_catalog.cc:458] T 00000000000000000000000000000000 P 60c579fec9964c88b77f838d9c13df36 [sys.catalog]: This master's current role is: LEADER
I20260809 22:55:49.478447 5260 sys_catalog.cc:455] T 00000000000000000000000000000000 P 60c579fec9964c88b77f838d9c13df36 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 60c579fec9964c88b77f838d9c13df36. Latest consensus state: current_term: 1 leader_uuid: "60c579fec9964c88b77f838d9c13df36" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60c579fec9964c88b77f838d9c13df36" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 46137 } } }
I20260809 22:55:49.479048 5260 sys_catalog.cc:458] T 00000000000000000000000000000000 P 60c579fec9964c88b77f838d9c13df36 [sys.catalog]: This master's current role is: LEADER
I20260809 22:55:49.480098 5267 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 22:55:49.488807 5267 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 22:55:49.497257 5267 catalog_manager.cc:1347] Generated new cluster ID: 85ec813c10da49099b9340d15162601d
I20260809 22:55:49.497352 5267 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 22:55:49.508414 5267 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 22:55:49.509907 5267 catalog_manager.cc:1504] Loading token signing keys...
I20260809 22:55:49.517777 5267 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 60c579fec9964c88b77f838d9c13df36: Generated new TSK 0
I20260809 22:55:49.518529 5267 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.113.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.113.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.113.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.129.keytab.
Entry for principal kudu/127.15.113.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.129.keytab.
I20260809 22:55:49.591634 15814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskItYPJR/build/asan/bin/kudu
/tmp/dist-test-taskItYPJR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315935617919-15814-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315935617919-15814-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315935617919-15814-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315935617919-15814-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.113.129:0
--local_ip_for_outbound_sockets=127.15.113.129
--webserver_interface=127.15.113.129
--webserver_port=0
--tserver_master_addrs=127.15.113.190:46137
--keytab_file=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.129.keytab
--principal=kudu/127.15.113.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.113.148:34295
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315935617919-15814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315935617919-15814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315935617919-15814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:55:49.925511 5289 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:55:49.925772 5289 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:55:49.925871 5289 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:55:49.932855 5289 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:55:49.932969 5289 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:55:49.933085 5289 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.113.129
I20260809 22:55:49.942134 5289 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.113.148:34295
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315935617919-15814-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315935617919-15814-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.129.keytab
--principal=kudu/127.15.113.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.113.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315935617919-15814-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.113.129
--webserver_port=0
--tserver_master_addrs=127.15.113.190:46137
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.113.129
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315935617919-15814-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:55:49.943266 5289 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:55:49.944489 5289 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:55:49.956708 5300 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:55:49.956774 5296 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:55:49.956807 5298 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:55:49.957561 5289 server_base.cc:1034] running on GCE node
I20260809 22:55:49.958150 5289 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:55:49.958978 5289 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:55:49.960171 5289 hybrid_clock.cc:648] HybridClock initialized: now 1786316149960129 us; error 32 us; skew 500 ppm
I20260809 22:55:49.960458 5289 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:55:49 dist-test-slave-ncp9 krb5kdc[5135](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316149, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.113.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:55:49.963627 5289 init.cc:377] Logged in from keytab as kudu/127.15.113.129@KRBTEST.COM (short username kudu)
I20260809 22:55:49.965230 5289 webserver.cc:460] Webserver started at http://127.15.113.129:45525/ using document root <none> and password file <none>
I20260809 22:55:49.966048 5289 fs_manager.cc:359] Metadata directory not provided
I20260809 22:55:49.966185 5289 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:55:49.966507 5289 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:55:49.970005 5289 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315935617919-15814-0/minicluster-data/ts-0/data/instance:
uuid: "e9491fd35ae64be7be6fa206b46d187a"
format_stamp: "Formatted at 2026-08-09 22:55:49 on dist-test-slave-ncp9"
server_key: "688d8c1cc62f3dd447fd01428a4507b3"
server_key_iv: "1f230b0c43cfd79a3fc22fe895743930"
server_key_version: "encryptionkey@0"
I20260809 22:55:49.970726 5289 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315935617919-15814-0/minicluster-data/ts-0/wal/instance:
uuid: "e9491fd35ae64be7be6fa206b46d187a"
format_stamp: "Formatted at 2026-08-09 22:55:49 on dist-test-slave-ncp9"
server_key: "688d8c1cc62f3dd447fd01428a4507b3"
server_key_iv: "1f230b0c43cfd79a3fc22fe895743930"
server_key_version: "encryptionkey@0"
I20260809 22:55:49.975575 5289 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260809 22:55:49.978888 5310 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:55:49.980170 5289 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 22:55:49.980302 5289 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315935617919-15814-0/minicluster-data/ts-0/data,/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315935617919-15814-0/minicluster-data/ts-0/wal
uuid: "e9491fd35ae64be7be6fa206b46d187a"
format_stamp: "Formatted at 2026-08-09 22:55:49 on dist-test-slave-ncp9"
server_key: "688d8c1cc62f3dd447fd01428a4507b3"
server_key_iv: "1f230b0c43cfd79a3fc22fe895743930"
server_key_version: "encryptionkey@0"
I20260809 22:55:49.980463 5289 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315935617919-15814-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315935617919-15814-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315935617919-15814-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:55:50.005724 5289 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:55:50.009485 5289 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:55:50.009723 5289 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:55:50.010617 5289 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:55:50.011878 5289 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:55:50.011950 5289 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:55:50.012037 5289 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:55:50.012079 5289 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:55:50.041776 5289 rpc_server.cc:307] RPC server started. Bound to: 127.15.113.129:34939
I20260809 22:55:50.041821 5495 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.113.129:34939 every 8 connection(s)
I20260809 22:55:50.043015 5289 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315935617919-15814-0/minicluster-data/ts-0/data/info.pb
I20260809 22:55:50.051467 15814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskItYPJR/build/asan/bin/kudu as pid 5289
I20260809 22:55:50.051657 15814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315935617919-15814-0/minicluster-data/ts-0/wal/instance
I20260809 22:55:50.052276 15814 external_mini_cluster.cc:1428] Setting key 42a7a636ec0517fe6dd72b68a06f2d99
Aug 09 22:55:50 dist-test-slave-ncp9 krb5kdc[5135](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316149, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.113.129@KRBTEST.COM for kudu/127.15.113.190@KRBTEST.COM
I20260809 22:55:50.062896 5499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:55:50.045454 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.113.129:60381 (local address 127.15.113.190:46137)
0809 22:55:50.045880 (+ 426us) server_negotiation.cc:206] Beginning negotiation
0809 22:55:50.045884 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 22:55:50.046920 (+ 1036us) server_negotiation.cc:407] Connection header received
0809 22:55:50.048710 (+ 1790us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:55:50.048716 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:55:50.048794 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:55:50.048907 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:55:50.050361 (+ 1454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:55:50.051065 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:55:50.052761 (+ 1696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:55:50.053303 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:55:50.053410 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:55:50.056876 (+ 3466us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:55:50.056882 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:55:50.056886 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:55:50.056944 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:55:50.059845 (+ 2901us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:55:50.060586 (+ 741us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:55:50.060592 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:55:50.060595 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:55:50.060749 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:55:50.061192 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:55:50.061196 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:55:50.061199 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:55:50.061502 (+ 303us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:55:50.061646 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0809 22:55:50.062361 (+ 715us) server_negotiation.cc:299] Negotiation successful
0809 22:55:50.062573 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":308,"thread_start_us":206,"threads_started":1}
I20260809 22:55:50.063832 5502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:55:50.046043 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:46137 (local address 127.15.113.129:60381)
0809 22:55:50.046768 (+ 725us) negotiation.cc:107] Waiting for socket to connect
0809 22:55:50.046808 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0809 22:55:50.048445 (+ 1637us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:55:50.049158 (+ 713us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:55:50.049165 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:55:50.049645 (+ 480us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:55:50.050196 (+ 551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:55:50.050214 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:55:50.051237 (+ 1023us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:55:50.051242 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:55:50.052602 (+ 1360us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:55:50.052621 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:55:50.053558 (+ 937us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:55:50.053566 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:55:50.053844 (+ 278us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:55:50.054553 (+ 709us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:55:50.054574 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:55:50.056669 (+ 2095us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:55:50.060007 (+ 3338us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:55:50.060014 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:55:50.060025 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:55:50.060463 (+ 438us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:55:50.060845 (+ 382us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:55:50.060849 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:55:50.060852 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:55:50.061085 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:55:50.061610 (+ 525us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:55:50.061617 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:55:50.061971 (+ 354us) client_negotiation.cc:769] Sending connection context
0809 22:55:50.062252 (+ 281us) client_negotiation.cc:240] Negotiation successful
0809 22:55:50.062711 (+ 459us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":491,"thread_start_us":268,"threads_started":1}
I20260809 22:55:50.065218 5496 heartbeater.cc:347] Connected to a master server at 127.15.113.190:46137
I20260809 22:55:50.065500 5496 heartbeater.cc:464] Registering TS with master...
I20260809 22:55:50.066200 5496 heartbeater.cc:511] Master 127.15.113.190:46137 requested a full tablet report, sending...
I20260809 22:55:50.068400 5200 ts_manager.cc:194] Registered new tserver with Master: e9491fd35ae64be7be6fa206b46d187a (127.15.113.129:34939)
I20260809 22:55:50.070298 5200 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.113.129@KRBTEST.COM'} at 127.15.113.129:60381
WARNING: no policy specified for kudu/127.15.113.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.113.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.113.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.130.keytab.
Entry for principal kudu/127.15.113.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.130.keytab.
I20260809 22:55:50.113529 15814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskItYPJR/build/asan/bin/kudu
/tmp/dist-test-taskItYPJR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315935617919-15814-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315935617919-15814-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315935617919-15814-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315935617919-15814-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.113.130:0
--local_ip_for_outbound_sockets=127.15.113.130
--webserver_interface=127.15.113.130
--webserver_port=0
--tserver_master_addrs=127.15.113.190:46137
--keytab_file=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.130.keytab
--principal=kudu/127.15.113.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.113.148:34295
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315935617919-15814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315935617919-15814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315935617919-15814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:55:50.438503 5505 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:55:50.438809 5505 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:55:50.438949 5505 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:55:50.450065 5505 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:55:50.450187 5505 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:55:50.450364 5505 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.113.130
I20260809 22:55:50.462002 5505 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.113.148:34295
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315935617919-15814-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315935617919-15814-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.130.keytab
--principal=kudu/127.15.113.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.113.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315935617919-15814-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.113.130
--webserver_port=0
--tserver_master_addrs=127.15.113.190:46137
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.113.130
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315935617919-15814-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:55:50.463121 5505 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:55:50.464313 5505 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:55:50.476725 5514 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:55:50.476699 5518 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:55:50.476750 5515 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:55:50.482311 5505 server_base.cc:1034] running on GCE node
I20260809 22:55:50.482879 5505 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:55:50.483793 5505 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:55:50.484988 5505 hybrid_clock.cc:648] HybridClock initialized: now 1786316150484941 us; error 39 us; skew 500 ppm
I20260809 22:55:50.485296 5505 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:55:50 dist-test-slave-ncp9 krb5kdc[5135](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316150, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.113.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:55:50.488255 5505 init.cc:377] Logged in from keytab as kudu/127.15.113.130@KRBTEST.COM (short username kudu)
I20260809 22:55:50.489724 5505 webserver.cc:460] Webserver started at http://127.15.113.130:40721/ using document root <none> and password file <none>
I20260809 22:55:50.490422 5505 fs_manager.cc:359] Metadata directory not provided
I20260809 22:55:50.490541 5505 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:55:50.490836 5505 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:55:50.493731 5505 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315935617919-15814-0/minicluster-data/ts-1/data/instance:
uuid: "6fb2492799864fb8b944c12e0aa53c0a"
format_stamp: "Formatted at 2026-08-09 22:55:50 on dist-test-slave-ncp9"
server_key: "3bd00e7d274a6999ef76a1d8237a5ab7"
server_key_iv: "29b30cacc051bf72a62afb639737ac6c"
server_key_version: "encryptionkey@0"
I20260809 22:55:50.494513 5505 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315935617919-15814-0/minicluster-data/ts-1/wal/instance:
uuid: "6fb2492799864fb8b944c12e0aa53c0a"
format_stamp: "Formatted at 2026-08-09 22:55:50 on dist-test-slave-ncp9"
server_key: "3bd00e7d274a6999ef76a1d8237a5ab7"
server_key_iv: "29b30cacc051bf72a62afb639737ac6c"
server_key_version: "encryptionkey@0"
I20260809 22:55:50.499531 5505 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260809 22:55:50.502872 5530 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:55:50.504206 5505 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 22:55:50.504356 5505 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315935617919-15814-0/minicluster-data/ts-1/data,/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315935617919-15814-0/minicluster-data/ts-1/wal
uuid: "6fb2492799864fb8b944c12e0aa53c0a"
format_stamp: "Formatted at 2026-08-09 22:55:50 on dist-test-slave-ncp9"
server_key: "3bd00e7d274a6999ef76a1d8237a5ab7"
server_key_iv: "29b30cacc051bf72a62afb639737ac6c"
server_key_version: "encryptionkey@0"
I20260809 22:55:50.504539 5505 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315935617919-15814-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315935617919-15814-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315935617919-15814-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:55:50.520493 5505 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:55:50.524195 5505 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:55:50.524430 5505 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:55:50.525363 5505 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:55:50.526674 5505 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:55:50.526752 5505 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:55:50.526870 5505 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:55:50.526923 5505 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:55:50.554939 5505 rpc_server.cc:307] RPC server started. Bound to: 127.15.113.130:39003
I20260809 22:55:50.554981 5708 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.113.130:39003 every 8 connection(s)
I20260809 22:55:50.556180 5505 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315935617919-15814-0/minicluster-data/ts-1/data/info.pb
I20260809 22:55:50.566205 15814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskItYPJR/build/asan/bin/kudu as pid 5505
I20260809 22:55:50.566367 15814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315935617919-15814-0/minicluster-data/ts-1/wal/instance
I20260809 22:55:50.566936 15814 external_mini_cluster.cc:1428] Setting key 11fa24570d6043b3c55c8bf20950709d
Aug 09 22:55:50 dist-test-slave-ncp9 krb5kdc[5135](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316150, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.113.130@KRBTEST.COM for kudu/127.15.113.190@KRBTEST.COM
I20260809 22:55:50.575845 5499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:55:50.558554 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.113.130:43911 (local address 127.15.113.190:46137)
0809 22:55:50.558721 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0809 22:55:50.558726 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:55:50.559859 (+ 1133us) server_negotiation.cc:407] Connection header received
0809 22:55:50.561742 (+ 1883us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:55:50.561749 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:55:50.561832 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:55:50.561921 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:55:50.563392 (+ 1471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:55:50.564098 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:55:50.565846 (+ 1748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:55:50.566270 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:55:50.566380 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:55:50.570423 (+ 4043us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:55:50.570428 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:55:50.570431 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:55:50.570463 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:55:50.572877 (+ 2414us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:55:50.573699 (+ 822us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:55:50.573703 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:55:50.573705 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:55:50.573833 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:55:50.574307 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:55:50.574310 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:55:50.574312 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:55:50.574559 (+ 247us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:55:50.574675 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0809 22:55:50.575433 (+ 758us) server_negotiation.cc:299] Negotiation successful
0809 22:55:50.575599 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260809 22:55:50.577271 5712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:55:50.558998 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:46137 (local address 127.15.113.130:43911)
0809 22:55:50.559694 (+ 696us) negotiation.cc:107] Waiting for socket to connect
0809 22:55:50.559739 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0809 22:55:50.561469 (+ 1730us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:55:50.562175 (+ 706us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:55:50.562195 (+ 20us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:55:50.562677 (+ 482us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:55:50.563222 (+ 545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:55:50.563240 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:55:50.564259 (+ 1019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:55:50.564262 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:55:50.565690 (+ 1428us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:55:50.565703 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:55:50.566551 (+ 848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:55:50.566560 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:55:50.566872 (+ 312us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:55:50.567545 (+ 673us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:55:50.567573 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:55:50.570213 (+ 2640us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:55:50.573034 (+ 2821us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:55:50.573042 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:55:50.573058 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:55:50.573596 (+ 538us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:55:50.573928 (+ 332us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:55:50.573932 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:55:50.573936 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:55:50.574216 (+ 280us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:55:50.574676 (+ 460us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:55:50.574683 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:55:50.575082 (+ 399us) client_negotiation.cc:769] Sending connection context
0809 22:55:50.575384 (+ 302us) client_negotiation.cc:240] Negotiation successful
0809 22:55:50.575882 (+ 498us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":467,"thread_start_us":278,"threads_started":1}
I20260809 22:55:50.578691 5710 heartbeater.cc:347] Connected to a master server at 127.15.113.190:46137
I20260809 22:55:50.578959 5710 heartbeater.cc:464] Registering TS with master...
I20260809 22:55:50.579661 5710 heartbeater.cc:511] Master 127.15.113.190:46137 requested a full tablet report, sending...
I20260809 22:55:50.581131 5200 ts_manager.cc:194] Registered new tserver with Master: 6fb2492799864fb8b944c12e0aa53c0a (127.15.113.130:39003)
I20260809 22:55:50.582000 5200 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.113.130@KRBTEST.COM'} at 127.15.113.130:43911
WARNING: no policy specified for kudu/127.15.113.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.113.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.113.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.131.keytab.
Entry for principal kudu/127.15.113.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.131.keytab.
I20260809 22:55:50.625720 15814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskItYPJR/build/asan/bin/kudu
/tmp/dist-test-taskItYPJR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315935617919-15814-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315935617919-15814-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315935617919-15814-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315935617919-15814-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.113.131:0
--local_ip_for_outbound_sockets=127.15.113.131
--webserver_interface=127.15.113.131
--webserver_port=0
--tserver_master_addrs=127.15.113.190:46137
--keytab_file=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.131.keytab
--principal=kudu/127.15.113.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.113.148:34295
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315935617919-15814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315935617919-15814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315935617919-15814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:55:50.950222 5716 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:55:50.950501 5716 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:55:50.950629 5716 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:55:50.957579 5716 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:55:50.957685 5716 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:55:50.957834 5716 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.113.131
I20260809 22:55:50.967238 5716 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.113.148:34295
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315935617919-15814-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315935617919-15814-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.131.keytab
--principal=kudu/127.15.113.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.113.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315935617919-15814-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.113.131
--webserver_port=0
--tserver_master_addrs=127.15.113.190:46137
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.113.131
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315935617919-15814-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:55:50.968477 5716 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:55:50.969815 5716 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:55:50.982219 5723 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:55:50.982297 5728 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:55:50.982343 5725 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:55:50.983415 5716 server_base.cc:1034] running on GCE node
I20260809 22:55:50.984041 5716 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:55:50.985041 5716 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:55:50.986241 5716 hybrid_clock.cc:648] HybridClock initialized: now 1786316150986187 us; error 40 us; skew 500 ppm
I20260809 22:55:50.986527 5716 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:55:50 dist-test-slave-ncp9 krb5kdc[5135](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316150, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.113.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:55:50.989500 5716 init.cc:377] Logged in from keytab as kudu/127.15.113.131@KRBTEST.COM (short username kudu)
I20260809 22:55:50.990801 5716 webserver.cc:460] Webserver started at http://127.15.113.131:41869/ using document root <none> and password file <none>
I20260809 22:55:50.991441 5716 fs_manager.cc:359] Metadata directory not provided
I20260809 22:55:50.991559 5716 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:55:50.991839 5716 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:55:50.994699 5716 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315935617919-15814-0/minicluster-data/ts-2/data/instance:
uuid: "82971d28843442a7a9c8338a4ad220bb"
format_stamp: "Formatted at 2026-08-09 22:55:50 on dist-test-slave-ncp9"
server_key: "159f3a218e0536600f31a263f5e25b04"
server_key_iv: "0359d24e8ea6f74a0f00667a35524811"
server_key_version: "encryptionkey@0"
I20260809 22:55:50.995520 5716 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315935617919-15814-0/minicluster-data/ts-2/wal/instance:
uuid: "82971d28843442a7a9c8338a4ad220bb"
format_stamp: "Formatted at 2026-08-09 22:55:50 on dist-test-slave-ncp9"
server_key: "159f3a218e0536600f31a263f5e25b04"
server_key_iv: "0359d24e8ea6f74a0f00667a35524811"
server_key_version: "encryptionkey@0"
I20260809 22:55:51.000542 5716 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260809 22:55:51.003880 5737 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:55:51.005141 5716 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 22:55:51.005295 5716 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315935617919-15814-0/minicluster-data/ts-2/data,/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315935617919-15814-0/minicluster-data/ts-2/wal
uuid: "82971d28843442a7a9c8338a4ad220bb"
format_stamp: "Formatted at 2026-08-09 22:55:50 on dist-test-slave-ncp9"
server_key: "159f3a218e0536600f31a263f5e25b04"
server_key_iv: "0359d24e8ea6f74a0f00667a35524811"
server_key_version: "encryptionkey@0"
I20260809 22:55:51.005472 5716 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315935617919-15814-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315935617919-15814-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315935617919-15814-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:55:51.021868 5716 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:55:51.025601 5716 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:55:51.025863 5716 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:55:51.026775 5716 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:55:51.028107 5716 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:55:51.028195 5716 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:55:51.028319 5716 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:55:51.028371 5716 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:55:51.058799 5716 rpc_server.cc:307] RPC server started. Bound to: 127.15.113.131:36817
I20260809 22:55:51.058842 5914 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.113.131:36817 every 8 connection(s)
I20260809 22:55:51.060047 5716 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315935617919-15814-0/minicluster-data/ts-2/data/info.pb
I20260809 22:55:51.069835 15814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskItYPJR/build/asan/bin/kudu as pid 5716
I20260809 22:55:51.070003 15814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315935617919-15814-0/minicluster-data/ts-2/wal/instance
I20260809 22:55:51.070461 15814 external_mini_cluster.cc:1428] Setting key 3fb5100ba42f1c4a251b8849dfc8712e
Aug 09 22:55:51 dist-test-slave-ncp9 krb5kdc[5135](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316150, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.113.131@KRBTEST.COM for kudu/127.15.113.190@KRBTEST.COM
I20260809 22:55:51.073598 5496 heartbeater.cc:503] Master 127.15.113.190:46137 was elected leader, sending a full tablet report...
I20260809 22:55:51.079259 5499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:55:51.062530 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.113.131:48815 (local address 127.15.113.190:46137)
0809 22:55:51.062741 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0809 22:55:51.062748 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:55:51.063906 (+ 1158us) server_negotiation.cc:407] Connection header received
0809 22:55:51.065703 (+ 1797us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:55:51.065707 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:55:51.065804 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:55:51.065890 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:55:51.067263 (+ 1373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:55:51.067996 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:55:51.069505 (+ 1509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:55:51.069891 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:55:51.070001 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:55:51.073663 (+ 3662us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:55:51.073669 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:55:51.073675 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:55:51.073714 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:55:51.076607 (+ 2893us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:55:51.077315 (+ 708us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:55:51.077319 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:55:51.077321 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:55:51.077450 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:55:51.077839 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:55:51.077844 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:55:51.077847 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:55:51.078170 (+ 323us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:55:51.078286 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0809 22:55:51.078843 (+ 557us) server_negotiation.cc:299] Negotiation successful
0809 22:55:51.078999 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260809 22:55:51.080121 5919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:55:51.063051 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:46137 (local address 127.15.113.131:48815)
0809 22:55:51.063752 (+ 701us) negotiation.cc:107] Waiting for socket to connect
0809 22:55:51.063796 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 22:55:51.065457 (+ 1661us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:55:51.066080 (+ 623us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:55:51.066087 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:55:51.066561 (+ 474us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:55:51.067129 (+ 568us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:55:51.067147 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:55:51.068108 (+ 961us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:55:51.068112 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:55:51.069400 (+ 1288us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:55:51.069413 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:55:51.070113 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:55:51.070120 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:55:51.070347 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:55:51.070991 (+ 644us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:55:51.071016 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:55:51.073410 (+ 2394us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:55:51.076752 (+ 3342us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:55:51.076758 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:55:51.076769 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:55:51.077221 (+ 452us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:55:51.077533 (+ 312us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:55:51.077537 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:55:51.077539 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:55:51.077740 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:55:51.078265 (+ 525us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:55:51.078270 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:55:51.078553 (+ 283us) client_negotiation.cc:769] Sending connection context
0809 22:55:51.078771 (+ 218us) client_negotiation.cc:240] Negotiation successful
0809 22:55:51.079116 (+ 345us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":461,"thread_start_us":261,"threads_started":1}
I20260809 22:55:51.081362 5915 heartbeater.cc:347] Connected to a master server at 127.15.113.190:46137
I20260809 22:55:51.081629 5915 heartbeater.cc:464] Registering TS with master...
I20260809 22:55:51.082329 5915 heartbeater.cc:511] Master 127.15.113.190:46137 requested a full tablet report, sending...
I20260809 22:55:51.083561 5200 ts_manager.cc:194] Registered new tserver with Master: 82971d28843442a7a9c8338a4ad220bb (127.15.113.131:36817)
I20260809 22:55:51.084425 5200 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.113.131@KRBTEST.COM'} at 127.15.113.131:48815
I20260809 22:55:51.086009 15814 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 22:55:51.101306 5499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:55:51.090261 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58076 (local address 127.15.113.190:46137)
0809 22:55:51.090418 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0809 22:55:51.090422 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 22:55:51.090824 (+ 402us) server_negotiation.cc:407] Connection header received
0809 22:55:51.090953 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:55:51.090956 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:55:51.091008 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:55:51.091081 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:55:51.092342 (+ 1261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:55:51.093065 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:55:51.094316 (+ 1251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:55:51.094739 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:55:51.094830 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:55:51.097073 (+ 2243us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:55:51.097078 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:55:51.097081 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:55:51.097107 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:55:51.099223 (+ 2116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:55:51.099770 (+ 547us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:55:51.099773 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:55:51.099776 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:55:51.099892 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:55:51.100180 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:55:51.100183 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:55:51.100185 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:55:51.100461 (+ 276us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:55:51.100626 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0809 22:55:51.100853 (+ 227us) server_negotiation.cc:299] Negotiation successful
0809 22:55:51.101038 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260809 22:55:51.104969 5200 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58076:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 22:55:51.111739 5200 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 22:55:51.142131 5941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:55:51.123007 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49228 (local address 127.15.113.130:39003)
0809 22:55:51.134906 (+ 11899us) server_negotiation.cc:206] Beginning negotiation
0809 22:55:51.134915 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 22:55:51.134946 (+ 31us) server_negotiation.cc:407] Connection header received
0809 22:55:51.135057 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:55:51.135064 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:55:51.135322 (+ 258us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:55:51.135547 (+ 225us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:55:51.136206 (+ 659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:55:51.137795 (+ 1589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:55:51.139777 (+ 1982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:55:51.140964 (+ 1187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:55:51.141108 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:55:51.141293 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0809 22:55:51.141637 (+ 344us) server_negotiation.cc:299] Negotiation successful
0809 22:55:51.141802 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11738,"thread_start_us":11586,"threads_started":1}
I20260809 22:55:51.142153 5937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:55:51.122762 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.130:39003 (local address 127.0.0.1:49228)
0809 22:55:51.134228 (+ 11466us) negotiation.cc:107] Waiting for socket to connect
0809 22:55:51.134288 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0809 22:55:51.134505 (+ 217us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:55:51.135545 (+ 1040us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:55:51.135552 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:55:51.135582 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:55:51.135971 (+ 389us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:55:51.135990 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:55:51.137940 (+ 1950us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:55:51.137949 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:55:51.139579 (+ 1630us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:55:51.139594 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:55:51.141189 (+ 1595us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:55:51.141195 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:55:51.141409 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:55:51.141452 (+ 43us) client_negotiation.cc:769] Sending connection context
0809 22:55:51.141565 (+ 113us) client_negotiation.cc:240] Negotiation successful
0809 22:55:51.141758 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11151,"thread_start_us":12997,"threads_started":1}
I20260809 22:55:51.144759 5936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:55:51.123015 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.131:36817 (local address 127.0.0.1:50060)
0809 22:55:51.136218 (+ 13203us) negotiation.cc:107] Waiting for socket to connect
0809 22:55:51.136241 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0809 22:55:51.136369 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:55:51.137344 (+ 975us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:55:51.137349 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:55:51.137373 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:55:51.137800 (+ 427us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:55:51.137812 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:55:51.139783 (+ 1971us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:55:51.139789 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:55:51.142056 (+ 2267us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:55:51.142069 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:55:51.143906 (+ 1837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:55:51.143911 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:55:51.144062 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:55:51.144098 (+ 36us) client_negotiation.cc:769] Sending connection context
0809 22:55:51.144208 (+ 110us) client_negotiation.cc:240] Negotiation successful
0809 22:55:51.144330 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13047,"mutex_wait_us":5,"thread_start_us":11685,"threads_started":1}
I20260809 22:55:51.144780 5944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:55:51.124088 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50060 (local address 127.15.113.131:36817)
0809 22:55:51.135915 (+ 11827us) server_negotiation.cc:206] Beginning negotiation
0809 22:55:51.135925 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 22:55:51.136513 (+ 588us) server_negotiation.cc:407] Connection header received
0809 22:55:51.136632 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:55:51.136641 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:55:51.136878 (+ 237us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:55:51.137075 (+ 197us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:55:51.138005 (+ 930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:55:51.139617 (+ 1612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:55:51.142269 (+ 2652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:55:51.143672 (+ 1403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:55:51.143857 (+ 185us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:55:51.144020 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0809 22:55:51.144262 (+ 242us) server_negotiation.cc:299] Negotiation successful
0809 22:55:51.144423 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11650,"thread_start_us":11489,"threads_started":1}
I20260809 22:55:51.145344 5945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:55:51.124574 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57686 (local address 127.15.113.129:34939)
0809 22:55:51.135416 (+ 10842us) server_negotiation.cc:206] Beginning negotiation
0809 22:55:51.135425 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 22:55:51.136672 (+ 1247us) server_negotiation.cc:407] Connection header received
0809 22:55:51.136789 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:55:51.136796 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:55:51.137060 (+ 264us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:55:51.137270 (+ 210us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:55:51.138189 (+ 919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:55:51.139746 (+ 1557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:55:51.142269 (+ 2523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:55:51.143672 (+ 1403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:55:51.143857 (+ 185us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:55:51.144021 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0809 22:55:51.144869 (+ 848us) server_negotiation.cc:299] Negotiation successful
0809 22:55:51.145053 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10675,"thread_start_us":10515,"threads_started":1}
I20260809 22:55:51.145761 5942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:55:51.123354 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.129:34939 (local address 127.0.0.1:57686)
0809 22:55:51.136404 (+ 13050us) negotiation.cc:107] Waiting for socket to connect
0809 22:55:51.136421 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 22:55:51.136539 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:55:51.137498 (+ 959us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:55:51.137501 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:55:51.137521 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:55:51.137964 (+ 443us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:55:51.137975 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:55:51.139909 (+ 1934us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:55:51.139913 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:55:51.142053 (+ 2140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:55:51.142069 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:55:51.144158 (+ 2089us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:55:51.144163 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:55:51.144325 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:55:51.144358 (+ 33us) client_negotiation.cc:769] Sending connection context
0809 22:55:51.144446 (+ 88us) client_negotiation.cc:240] Negotiation successful
0809 22:55:51.144566 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12910,"mutex_wait_us":27,"thread_start_us":12752,"threads_started":1}
I20260809 22:55:51.146440 5611 tablet_service.cc:1467] Processing CreateTablet for tablet 48b6146b41994239894fcbef8dc8966a (DEFAULT_TABLE table=test-table [id=77c3b6939a4a4ceeb50b8a8b0e30c43a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:55:51.147893 5611 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 48b6146b41994239894fcbef8dc8966a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:55:51.148509 5813 tablet_service.cc:1467] Processing CreateTablet for tablet 48b6146b41994239894fcbef8dc8966a (DEFAULT_TABLE table=test-table [id=77c3b6939a4a4ceeb50b8a8b0e30c43a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:55:51.150007 5813 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 48b6146b41994239894fcbef8dc8966a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:55:51.150421 5384 tablet_service.cc:1467] Processing CreateTablet for tablet 48b6146b41994239894fcbef8dc8966a (DEFAULT_TABLE table=test-table [id=77c3b6939a4a4ceeb50b8a8b0e30c43a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:55:51.151732 5384 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 48b6146b41994239894fcbef8dc8966a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:55:51.159077 5947 tablet_bootstrap.cc:492] T 48b6146b41994239894fcbef8dc8966a P 6fb2492799864fb8b944c12e0aa53c0a: Bootstrap starting.
I20260809 22:55:51.162196 5947 tablet_bootstrap.cc:654] T 48b6146b41994239894fcbef8dc8966a P 6fb2492799864fb8b944c12e0aa53c0a: Neither blocks nor log segments found. Creating new log.
I20260809 22:55:51.163172 5947 log.cc:824] T 48b6146b41994239894fcbef8dc8966a P 6fb2492799864fb8b944c12e0aa53c0a: Log is configured to *not* fsync() on all Append() calls
I20260809 22:55:51.164132 5948 tablet_bootstrap.cc:492] T 48b6146b41994239894fcbef8dc8966a P 82971d28843442a7a9c8338a4ad220bb: Bootstrap starting.
I20260809 22:55:51.166366 5951 tablet_bootstrap.cc:492] T 48b6146b41994239894fcbef8dc8966a P e9491fd35ae64be7be6fa206b46d187a: Bootstrap starting.
I20260809 22:55:51.167783 5948 tablet_bootstrap.cc:654] T 48b6146b41994239894fcbef8dc8966a P 82971d28843442a7a9c8338a4ad220bb: Neither blocks nor log segments found. Creating new log.
I20260809 22:55:51.168048 5947 tablet_bootstrap.cc:492] T 48b6146b41994239894fcbef8dc8966a P 6fb2492799864fb8b944c12e0aa53c0a: No bootstrap required, opened a new log
I20260809 22:55:51.168345 5947 ts_tablet_manager.cc:1397] T 48b6146b41994239894fcbef8dc8966a P 6fb2492799864fb8b944c12e0aa53c0a: Time spent bootstrapping tablet: real 0.009s user 0.000s sys 0.006s
I20260809 22:55:51.169023 5951 tablet_bootstrap.cc:654] T 48b6146b41994239894fcbef8dc8966a P e9491fd35ae64be7be6fa206b46d187a: Neither blocks nor log segments found. Creating new log.
I20260809 22:55:51.169006 5948 log.cc:824] T 48b6146b41994239894fcbef8dc8966a P 82971d28843442a7a9c8338a4ad220bb: Log is configured to *not* fsync() on all Append() calls
I20260809 22:55:51.170122 5951 log.cc:824] T 48b6146b41994239894fcbef8dc8966a P e9491fd35ae64be7be6fa206b46d187a: Log is configured to *not* fsync() on all Append() calls
I20260809 22:55:51.171943 5948 tablet_bootstrap.cc:492] T 48b6146b41994239894fcbef8dc8966a P 82971d28843442a7a9c8338a4ad220bb: No bootstrap required, opened a new log
I20260809 22:55:51.172216 5948 ts_tablet_manager.cc:1397] T 48b6146b41994239894fcbef8dc8966a P 82971d28843442a7a9c8338a4ad220bb: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260809 22:55:51.172474 5951 tablet_bootstrap.cc:492] T 48b6146b41994239894fcbef8dc8966a P e9491fd35ae64be7be6fa206b46d187a: No bootstrap required, opened a new log
I20260809 22:55:51.172763 5951 ts_tablet_manager.cc:1397] T 48b6146b41994239894fcbef8dc8966a P e9491fd35ae64be7be6fa206b46d187a: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.006s
I20260809 22:55:51.178259 5947 raft_consensus.cc:348] T 48b6146b41994239894fcbef8dc8966a P 6fb2492799864fb8b944c12e0aa53c0a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fb2492799864fb8b944c12e0aa53c0a" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 39003 } } peers { permanent_uuid: "82971d28843442a7a9c8338a4ad220bb" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 36817 } } peers { permanent_uuid: "e9491fd35ae64be7be6fa206b46d187a" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 34939 } }
I20260809 22:55:51.178683 5947 raft_consensus.cc:374] T 48b6146b41994239894fcbef8dc8966a P 6fb2492799864fb8b944c12e0aa53c0a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:55:51.178785 5947 raft_consensus.cc:729] T 48b6146b41994239894fcbef8dc8966a P 6fb2492799864fb8b944c12e0aa53c0a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fb2492799864fb8b944c12e0aa53c0a, State: Initialized, Role: FOLLOWER
I20260809 22:55:51.179380 5947 consensus_queue.cc:260] T 48b6146b41994239894fcbef8dc8966a P 6fb2492799864fb8b944c12e0aa53c0a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fb2492799864fb8b944c12e0aa53c0a" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 39003 } } peers { permanent_uuid: "82971d28843442a7a9c8338a4ad220bb" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 36817 } } peers { permanent_uuid: "e9491fd35ae64be7be6fa206b46d187a" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 34939 } }
I20260809 22:55:51.180480 5710 heartbeater.cc:503] Master 127.15.113.190:46137 was elected leader, sending a full tablet report...
I20260809 22:55:51.180614 5951 raft_consensus.cc:348] T 48b6146b41994239894fcbef8dc8966a P e9491fd35ae64be7be6fa206b46d187a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fb2492799864fb8b944c12e0aa53c0a" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 39003 } } peers { permanent_uuid: "82971d28843442a7a9c8338a4ad220bb" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 36817 } } peers { permanent_uuid: "e9491fd35ae64be7be6fa206b46d187a" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 34939 } }
I20260809 22:55:51.180956 5951 raft_consensus.cc:374] T 48b6146b41994239894fcbef8dc8966a P e9491fd35ae64be7be6fa206b46d187a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:55:51.181057 5951 raft_consensus.cc:729] T 48b6146b41994239894fcbef8dc8966a P e9491fd35ae64be7be6fa206b46d187a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9491fd35ae64be7be6fa206b46d187a, State: Initialized, Role: FOLLOWER
I20260809 22:55:51.181107 5947 ts_tablet_manager.cc:1428] T 48b6146b41994239894fcbef8dc8966a P 6fb2492799864fb8b944c12e0aa53c0a: Time spent starting tablet: real 0.013s user 0.014s sys 0.001s
I20260809 22:55:51.181589 5951 consensus_queue.cc:260] T 48b6146b41994239894fcbef8dc8966a P e9491fd35ae64be7be6fa206b46d187a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fb2492799864fb8b944c12e0aa53c0a" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 39003 } } peers { permanent_uuid: "82971d28843442a7a9c8338a4ad220bb" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 36817 } } peers { permanent_uuid: "e9491fd35ae64be7be6fa206b46d187a" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 34939 } }
I20260809 22:55:51.183223 5951 ts_tablet_manager.cc:1428] T 48b6146b41994239894fcbef8dc8966a P e9491fd35ae64be7be6fa206b46d187a: Time spent starting tablet: real 0.010s user 0.009s sys 0.000s
I20260809 22:55:51.183118 5948 raft_consensus.cc:348] T 48b6146b41994239894fcbef8dc8966a P 82971d28843442a7a9c8338a4ad220bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fb2492799864fb8b944c12e0aa53c0a" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 39003 } } peers { permanent_uuid: "82971d28843442a7a9c8338a4ad220bb" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 36817 } } peers { permanent_uuid: "e9491fd35ae64be7be6fa206b46d187a" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 34939 } }
I20260809 22:55:51.183561 5948 raft_consensus.cc:374] T 48b6146b41994239894fcbef8dc8966a P 82971d28843442a7a9c8338a4ad220bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:55:51.183686 5948 raft_consensus.cc:729] T 48b6146b41994239894fcbef8dc8966a P 82971d28843442a7a9c8338a4ad220bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 82971d28843442a7a9c8338a4ad220bb, State: Initialized, Role: FOLLOWER
I20260809 22:55:51.184684 5948 consensus_queue.cc:260] T 48b6146b41994239894fcbef8dc8966a P 82971d28843442a7a9c8338a4ad220bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fb2492799864fb8b944c12e0aa53c0a" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 39003 } } peers { permanent_uuid: "82971d28843442a7a9c8338a4ad220bb" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 36817 } } peers { permanent_uuid: "e9491fd35ae64be7be6fa206b46d187a" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 34939 } }
I20260809 22:55:51.185844 5915 heartbeater.cc:503] Master 127.15.113.190:46137 was elected leader, sending a full tablet report...
I20260809 22:55:51.186486 5948 ts_tablet_manager.cc:1428] T 48b6146b41994239894fcbef8dc8966a P 82971d28843442a7a9c8338a4ad220bb: Time spent starting tablet: real 0.014s user 0.014s sys 0.000s
W20260809 22:55:51.295279 5497 tablet.cc:2369] T 48b6146b41994239894fcbef8dc8966a P e9491fd35ae64be7be6fa206b46d187a: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 22:55:51.308032 5711 tablet.cc:2369] T 48b6146b41994239894fcbef8dc8966a P 6fb2492799864fb8b944c12e0aa53c0a: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 22:55:51.311486 5918 tablet.cc:2369] T 48b6146b41994239894fcbef8dc8966a P 82971d28843442a7a9c8338a4ad220bb: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 22:55:51.312817 5959 raft_consensus.cc:482] T 48b6146b41994239894fcbef8dc8966a P 82971d28843442a7a9c8338a4ad220bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 22:55:51.313084 5959 raft_consensus.cc:504] T 48b6146b41994239894fcbef8dc8966a P 82971d28843442a7a9c8338a4ad220bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fb2492799864fb8b944c12e0aa53c0a" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 39003 } } peers { permanent_uuid: "82971d28843442a7a9c8338a4ad220bb" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 36817 } } peers { permanent_uuid: "e9491fd35ae64be7be6fa206b46d187a" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 34939 } }
I20260809 22:55:51.314726 5959 leader_election.cc:288] T 48b6146b41994239894fcbef8dc8966a P 82971d28843442a7a9c8338a4ad220bb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6fb2492799864fb8b944c12e0aa53c0a (127.15.113.130:39003), e9491fd35ae64be7be6fa206b46d187a (127.15.113.129:34939)
I20260809 22:55:51.321452 5941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:55:51.315012 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.113.131:52319 (local address 127.15.113.130:39003)
0809 22:55:51.315229 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0809 22:55:51.315235 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:55:51.315402 (+ 167us) server_negotiation.cc:407] Connection header received
0809 22:55:51.315678 (+ 276us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:55:51.315684 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:55:51.315764 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:55:51.315855 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:55:51.316313 (+ 458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:55:51.317338 (+ 1025us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:55:51.319562 (+ 2224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:55:51.320625 (+ 1063us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:55:51.320784 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:55:51.320857 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0809 22:55:51.321097 (+ 240us) server_negotiation.cc:299] Negotiation successful
0809 22:55:51.321241 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260809 22:55:51.321455 5919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:55:51.315088 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.130:39003 (local address 127.15.113.131:52319)
0809 22:55:51.315295 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0809 22:55:51.315316 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0809 22:55:51.315453 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:55:51.315898 (+ 445us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:55:51.315907 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:55:51.315937 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:55:51.316165 (+ 228us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:55:51.316177 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:55:51.317462 (+ 1285us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:55:51.317466 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:55:51.319395 (+ 1929us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:55:51.319412 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:55:51.320769 (+ 1357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:55:51.320774 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:55:51.320950 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:55:51.320978 (+ 28us) client_negotiation.cc:769] Sending connection context
0809 22:55:51.321047 (+ 69us) client_negotiation.cc:240] Negotiation successful
0809 22:55:51.321156 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":55,"mutex_wait_us":3}
I20260809 22:55:51.322335 5945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:55:51.315475 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.113.131:33653 (local address 127.15.113.129:34939)
0809 22:55:51.315669 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0809 22:55:51.315675 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:55:51.316024 (+ 349us) server_negotiation.cc:407] Connection header received
0809 22:55:51.316172 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:55:51.316175 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:55:51.316236 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:55:51.316321 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:55:51.316797 (+ 476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:55:51.317709 (+ 912us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:55:51.319880 (+ 2171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:55:51.321081 (+ 1201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:55:51.321238 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:55:51.321304 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0809 22:55:51.321959 (+ 655us) server_negotiation.cc:299] Negotiation successful
0809 22:55:51.322089 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260809 22:55:51.322391 5961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:55:51.315189 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.129:34939 (local address 127.15.113.131:33653)
0809 22:55:51.315916 (+ 727us) negotiation.cc:107] Waiting for socket to connect
0809 22:55:51.315938 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0809 22:55:51.316059 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:55:51.316348 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:55:51.316351 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:55:51.316371 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:55:51.316637 (+ 266us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:55:51.316645 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:55:51.317842 (+ 1197us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:55:51.317847 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:55:51.319741 (+ 1894us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:55:51.319752 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:55:51.321285 (+ 1533us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:55:51.321292 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:55:51.321461 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:55:51.321492 (+ 31us) client_negotiation.cc:769] Sending connection context
0809 22:55:51.321906 (+ 414us) client_negotiation.cc:240] Negotiation successful
0809 22:55:51.322023 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":615,"mutex_wait_us":24,"thread_start_us":395,"threads_started":1}
I20260809 22:55:51.322510 5638 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "48b6146b41994239894fcbef8dc8966a" candidate_uuid: "82971d28843442a7a9c8338a4ad220bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fb2492799864fb8b944c12e0aa53c0a" is_pre_election: true
I20260809 22:55:51.323110 5638 raft_consensus.cc:2454] T 48b6146b41994239894fcbef8dc8966a P 6fb2492799864fb8b944c12e0aa53c0a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 82971d28843442a7a9c8338a4ad220bb in term 0.
I20260809 22:55:51.323549 5425 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "48b6146b41994239894fcbef8dc8966a" candidate_uuid: "82971d28843442a7a9c8338a4ad220bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9491fd35ae64be7be6fa206b46d187a" is_pre_election: true
I20260809 22:55:51.323777 5739 leader_election.cc:302] T 48b6146b41994239894fcbef8dc8966a P 82971d28843442a7a9c8338a4ad220bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6fb2492799864fb8b944c12e0aa53c0a, 82971d28843442a7a9c8338a4ad220bb; no voters:
I20260809 22:55:51.324066 5425 raft_consensus.cc:2454] T 48b6146b41994239894fcbef8dc8966a P e9491fd35ae64be7be6fa206b46d187a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 82971d28843442a7a9c8338a4ad220bb in term 0.
I20260809 22:55:51.324151 5959 raft_consensus.cc:2781] T 48b6146b41994239894fcbef8dc8966a P 82971d28843442a7a9c8338a4ad220bb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 22:55:51.324290 5959 raft_consensus.cc:482] T 48b6146b41994239894fcbef8dc8966a P 82971d28843442a7a9c8338a4ad220bb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 22:55:51.324396 5959 raft_consensus.cc:3037] T 48b6146b41994239894fcbef8dc8966a P 82971d28843442a7a9c8338a4ad220bb [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:55:51.326719 5959 raft_consensus.cc:504] T 48b6146b41994239894fcbef8dc8966a P 82971d28843442a7a9c8338a4ad220bb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fb2492799864fb8b944c12e0aa53c0a" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 39003 } } peers { permanent_uuid: "82971d28843442a7a9c8338a4ad220bb" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 36817 } } peers { permanent_uuid: "e9491fd35ae64be7be6fa206b46d187a" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 34939 } }
I20260809 22:55:51.327271 5959 leader_election.cc:288] T 48b6146b41994239894fcbef8dc8966a P 82971d28843442a7a9c8338a4ad220bb [CANDIDATE]: Term 1 election: Requested vote from peers 6fb2492799864fb8b944c12e0aa53c0a (127.15.113.130:39003), e9491fd35ae64be7be6fa206b46d187a (127.15.113.129:34939)
I20260809 22:55:51.327626 5638 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "48b6146b41994239894fcbef8dc8966a" candidate_uuid: "82971d28843442a7a9c8338a4ad220bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fb2492799864fb8b944c12e0aa53c0a"
I20260809 22:55:51.327713 5425 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "48b6146b41994239894fcbef8dc8966a" candidate_uuid: "82971d28843442a7a9c8338a4ad220bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9491fd35ae64be7be6fa206b46d187a"
I20260809 22:55:51.327795 5638 raft_consensus.cc:3037] T 48b6146b41994239894fcbef8dc8966a P 6fb2492799864fb8b944c12e0aa53c0a [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:55:51.327886 5425 raft_consensus.cc:3037] T 48b6146b41994239894fcbef8dc8966a P e9491fd35ae64be7be6fa206b46d187a [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:55:51.330092 5638 raft_consensus.cc:2454] T 48b6146b41994239894fcbef8dc8966a P 6fb2492799864fb8b944c12e0aa53c0a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 82971d28843442a7a9c8338a4ad220bb in term 1.
I20260809 22:55:51.330175 5425 raft_consensus.cc:2454] T 48b6146b41994239894fcbef8dc8966a P e9491fd35ae64be7be6fa206b46d187a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 82971d28843442a7a9c8338a4ad220bb in term 1.
I20260809 22:55:51.330444 5739 leader_election.cc:302] T 48b6146b41994239894fcbef8dc8966a P 82971d28843442a7a9c8338a4ad220bb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6fb2492799864fb8b944c12e0aa53c0a, 82971d28843442a7a9c8338a4ad220bb; no voters:
I20260809 22:55:51.330734 5959 raft_consensus.cc:2781] T 48b6146b41994239894fcbef8dc8966a P 82971d28843442a7a9c8338a4ad220bb [term 1 FOLLOWER]: Leader election won for term 1
I20260809 22:55:51.331125 5959 raft_consensus.cc:686] T 48b6146b41994239894fcbef8dc8966a P 82971d28843442a7a9c8338a4ad220bb [term 1 LEADER]: Becoming Leader. State: Replica: 82971d28843442a7a9c8338a4ad220bb, State: Running, Role: LEADER
I20260809 22:55:51.331612 5959 consensus_queue.cc:237] T 48b6146b41994239894fcbef8dc8966a P 82971d28843442a7a9c8338a4ad220bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fb2492799864fb8b944c12e0aa53c0a" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 39003 } } peers { permanent_uuid: "82971d28843442a7a9c8338a4ad220bb" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 36817 } } peers { permanent_uuid: "e9491fd35ae64be7be6fa206b46d187a" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 34939 } }
I20260809 22:55:51.335783 5199 catalog_manager.cc:5495] T 48b6146b41994239894fcbef8dc8966a P 82971d28843442a7a9c8338a4ad220bb reported cstate change: term changed from 0 to 1, leader changed from <none> to 82971d28843442a7a9c8338a4ad220bb (127.15.113.131). New cstate: current_term: 1 leader_uuid: "82971d28843442a7a9c8338a4ad220bb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fb2492799864fb8b944c12e0aa53c0a" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 39003 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "82971d28843442a7a9c8338a4ad220bb" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 36817 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e9491fd35ae64be7be6fa206b46d187a" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 34939 } health_report { overall_health: UNKNOWN } } }
I20260809 22:55:51.379885 5944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:55:51.374765 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50070 (local address 127.15.113.131:36817)
0809 22:55:51.374983 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0809 22:55:51.374988 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:55:51.375007 (+ 19us) server_negotiation.cc:407] Connection header received
0809 22:55:51.375193 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:55:51.375197 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:55:51.375280 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:55:51.375398 (+ 118us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 22:55:51.375686 (+ 288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:55:51.376421 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:55:51.377962 (+ 1541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:55:51.378356 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:55:51.378445 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:55:51.378877 (+ 432us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 22:55:51.379384 (+ 507us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 22:55:51.379488 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0809 22:55:51.379560 (+ 72us) server_negotiation.cc:299] Negotiation successful
0809 22:55:51.379628 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260809 22:55:51.387806 5425 raft_consensus.cc:1261] T 48b6146b41994239894fcbef8dc8966a P e9491fd35ae64be7be6fa206b46d187a [term 1 FOLLOWER]: Refusing update from remote peer 82971d28843442a7a9c8338a4ad220bb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 22:55:51.387815 5638 raft_consensus.cc:1261] T 48b6146b41994239894fcbef8dc8966a P 6fb2492799864fb8b944c12e0aa53c0a [term 1 FOLLOWER]: Refusing update from remote peer 82971d28843442a7a9c8338a4ad220bb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 22:55:51.388931 5963 consensus_queue.cc:1035] T 48b6146b41994239894fcbef8dc8966a P 82971d28843442a7a9c8338a4ad220bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "6fb2492799864fb8b944c12e0aa53c0a" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 22:55:51.389386 5959 consensus_queue.cc:1035] T 48b6146b41994239894fcbef8dc8966a P 82971d28843442a7a9c8338a4ad220bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9491fd35ae64be7be6fa206b46d187a" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 34939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 22:55:51.399931 5967 mvcc.cc:205] Tried to move back new op lower bound from 7316750956074958848 to 7316750955857551360. Current Snapshot: MvccSnapshot[applied={T|T < 7316750956074958848}]
I20260809 22:55:51.404804 5969 mvcc.cc:205] Tried to move back new op lower bound from 7316750956074958848 to 7316750955857551360. Current Snapshot: MvccSnapshot[applied={T|T < 7316750956074958848}]
I20260809 22:55:51.413713 5199 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58076:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 22:55:51.414005 5199 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58076:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 22:55:51.415064 5968 mvcc.cc:205] Tried to move back new op lower bound from 7316750956074958848 to 7316750955857551360. Current Snapshot: MvccSnapshot[applied={T|T < 7316750956074958848}]
I20260809 22:55:51.417938 5199 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 60c579fec9964c88b77f838d9c13df36: Sending DeleteTablet for 3 replicas of tablet 48b6146b41994239894fcbef8dc8966a
I20260809 22:55:51.419216 5611 tablet_service.cc:1514] Processing DeleteTablet for tablet 48b6146b41994239894fcbef8dc8966a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:55:51 UTC) from {username='kudu', principal='kudu/127.15.113.190@KRBTEST.COM'} at 127.0.0.1:49228
I20260809 22:55:51.419366 5813 tablet_service.cc:1514] Processing DeleteTablet for tablet 48b6146b41994239894fcbef8dc8966a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:55:51 UTC) from {username='kudu', principal='kudu/127.15.113.190@KRBTEST.COM'} at 127.0.0.1:50060
I20260809 22:55:51.419919 5812 tablet_service.cc:1514] Processing DeleteTablet for tablet 48b6146b41994239894fcbef8dc8966a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:55:51 UTC) from {username='kudu', principal='kudu/127.15.113.190@KRBTEST.COM'} at 127.0.0.1:50060
I20260809 22:55:51.420400 15814 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskItYPJR/build/asan/bin/kudu pid 5289
I20260809 22:55:51.420578 5384 tablet_service.cc:1514] Processing DeleteTablet for tablet 48b6146b41994239894fcbef8dc8966a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:55:51 UTC) from {username='kudu', principal='kudu/127.15.113.190@KRBTEST.COM'} at 127.0.0.1:57686
I20260809 22:55:51.428843 5987 tablet_replica.cc:330] T 48b6146b41994239894fcbef8dc8966a P e9491fd35ae64be7be6fa206b46d187a: stopping tablet replica
I20260809 22:55:51.429324 5987 raft_consensus.cc:2229] T 48b6146b41994239894fcbef8dc8966a P e9491fd35ae64be7be6fa206b46d187a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 22:55:51.429874 5987 raft_consensus.cc:2258] T 48b6146b41994239894fcbef8dc8966a P e9491fd35ae64be7be6fa206b46d187a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 22:55:51.430631 5984 tablet_replica.cc:330] T 48b6146b41994239894fcbef8dc8966a P 6fb2492799864fb8b944c12e0aa53c0a: stopping tablet replica
I20260809 22:55:51.431042 5984 raft_consensus.cc:2229] T 48b6146b41994239894fcbef8dc8966a P 6fb2492799864fb8b944c12e0aa53c0a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 22:55:51.431522 5984 raft_consensus.cc:2258] T 48b6146b41994239894fcbef8dc8966a P 6fb2492799864fb8b944c12e0aa53c0a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 22:55:51.433027 5985 tablet_replica.cc:330] T 48b6146b41994239894fcbef8dc8966a P 82971d28843442a7a9c8338a4ad220bb: stopping tablet replica
I20260809 22:55:51.433440 5985 raft_consensus.cc:2229] T 48b6146b41994239894fcbef8dc8966a P 82971d28843442a7a9c8338a4ad220bb [term 1 LEADER]: Raft consensus shutting down.
I20260809 22:55:51.433612 5984 ts_tablet_manager.cc:1905] T 48b6146b41994239894fcbef8dc8966a P 6fb2492799864fb8b944c12e0aa53c0a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 22:55:51.434132 5985 raft_consensus.cc:2258] T 48b6146b41994239894fcbef8dc8966a P 82971d28843442a7a9c8338a4ad220bb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 22:55:51.435434 5987 ts_tablet_manager.cc:1905] T 48b6146b41994239894fcbef8dc8966a P e9491fd35ae64be7be6fa206b46d187a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 22:55:51.437038 5985 ts_tablet_manager.cc:1905] T 48b6146b41994239894fcbef8dc8966a P 82971d28843442a7a9c8338a4ad220bb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 22:55:51.440315 5984 ts_tablet_manager.cc:1918] T 48b6146b41994239894fcbef8dc8966a P 6fb2492799864fb8b944c12e0aa53c0a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 22:55:51.440469 5984 log.cc:1196] T 48b6146b41994239894fcbef8dc8966a P 6fb2492799864fb8b944c12e0aa53c0a: Deleting WAL directory at /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315935617919-15814-0/minicluster-data/ts-1/wal/wals/48b6146b41994239894fcbef8dc8966a
I20260809 22:55:51.440964 5984 ts_tablet_manager.cc:1939] T 48b6146b41994239894fcbef8dc8966a P 6fb2492799864fb8b944c12e0aa53c0a: Deleting consensus metadata
I20260809 22:55:51.442427 5987 ts_tablet_manager.cc:1918] T 48b6146b41994239894fcbef8dc8966a P e9491fd35ae64be7be6fa206b46d187a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 22:55:51.442588 5177 catalog_manager.cc:4841] TS 6fb2492799864fb8b944c12e0aa53c0a (127.15.113.130:39003): tablet 48b6146b41994239894fcbef8dc8966a (table test-table [id=77c3b6939a4a4ceeb50b8a8b0e30c43a]) successfully deleted
I20260809 22:55:51.442673 5987 log.cc:1196] T 48b6146b41994239894fcbef8dc8966a P e9491fd35ae64be7be6fa206b46d187a: Deleting WAL directory at /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315935617919-15814-0/minicluster-data/ts-0/wal/wals/48b6146b41994239894fcbef8dc8966a
I20260809 22:55:51.443270 5987 ts_tablet_manager.cc:1939] T 48b6146b41994239894fcbef8dc8966a P e9491fd35ae64be7be6fa206b46d187a: Deleting consensus metadata
I20260809 22:55:51.443682 5985 ts_tablet_manager.cc:1918] T 48b6146b41994239894fcbef8dc8966a P 82971d28843442a7a9c8338a4ad220bb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 22:55:51.443861 5985 log.cc:1196] T 48b6146b41994239894fcbef8dc8966a P 82971d28843442a7a9c8338a4ad220bb: Deleting WAL directory at /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315935617919-15814-0/minicluster-data/ts-2/wal/wals/48b6146b41994239894fcbef8dc8966a
I20260809 22:55:51.444370 5985 ts_tablet_manager.cc:1939] T 48b6146b41994239894fcbef8dc8966a P 82971d28843442a7a9c8338a4ad220bb: Deleting consensus metadata
I20260809 22:55:51.445218 5180 catalog_manager.cc:4841] TS e9491fd35ae64be7be6fa206b46d187a (127.15.113.129:34939): tablet 48b6146b41994239894fcbef8dc8966a (table test-table [id=77c3b6939a4a4ceeb50b8a8b0e30c43a]) successfully deleted
I20260809 22:55:51.450101 5181 catalog_manager.cc:4841] TS 82971d28843442a7a9c8338a4ad220bb (127.15.113.131:36817): tablet 48b6146b41994239894fcbef8dc8966a (table test-table [id=77c3b6939a4a4ceeb50b8a8b0e30c43a]) successfully deleted
W20260809 22:55:51.450513 5181 catalog_manager.cc:4808] TS 82971d28843442a7a9c8338a4ad220bb (127.15.113.131:36817): delete failed for tablet 48b6146b41994239894fcbef8dc8966a because the tablet was not found. No further retry: Not found: Tablet not found: 48b6146b41994239894fcbef8dc8966a
Aug 09 22:55:51 dist-test-slave-ncp9 krb5kdc[5135](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316148, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.113.129@KRBTEST.COM
I20260809 22:55:51.462015 5945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:55:51.421614 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57702 (local address 127.15.113.129:34939)
0809 22:55:51.421780 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0809 22:55:51.421787 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:55:51.432339 (+ 10552us) server_negotiation.cc:407] Connection header received
0809 22:55:51.432545 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:55:51.432551 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:55:51.432644 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:55:51.432754 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:55:51.446269 (+ 13515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:55:51.447370 (+ 1101us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:55:51.449450 (+ 2080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:55:51.450080 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:55:51.450177 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:55:51.455559 (+ 5382us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:55:51.455573 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:55:51.455590 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:55:51.455638 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:55:51.459225 (+ 3587us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:55:51.459897 (+ 672us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:55:51.459906 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:55:51.459909 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:55:51.460070 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:55:51.460402 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:55:51.460406 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:55:51.460410 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:55:51.460847 (+ 437us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:55:51.461190 (+ 343us) server_negotiation.cc:1035] Waiting for connection context
0809 22:55:51.461416 (+ 226us) server_negotiation.cc:299] Negotiation successful
0809 22:55:51.461658 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260809 22:55:51.462661 5490 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:55:51.550936 15814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskItYPJR/build/asan/bin/kudu with pid 5289
I20260809 22:55:51.582530 15814 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskItYPJR/build/asan/bin/kudu pid 5505
Aug 09 22:55:51 dist-test-slave-ncp9 krb5kdc[5135](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316148, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.113.130@KRBTEST.COM
I20260809 22:55:51.601810 5941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:55:51.583263 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49236 (local address 127.15.113.130:39003)
0809 22:55:51.583758 (+ 495us) server_negotiation.cc:206] Beginning negotiation
0809 22:55:51.583763 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:55:51.583777 (+ 14us) server_negotiation.cc:407] Connection header received
0809 22:55:51.583826 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:55:51.583829 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:55:51.583892 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:55:51.583977 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:55:51.585309 (+ 1332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:55:51.586038 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:55:51.587411 (+ 1373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:55:51.587836 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:55:51.587948 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:55:51.596237 (+ 8289us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:55:51.596250 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:55:51.596265 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:55:51.596301 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:55:51.599060 (+ 2759us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:55:51.599697 (+ 637us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:55:51.599705 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:55:51.599708 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:55:51.599832 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:55:51.600206 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:55:51.600210 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:55:51.600212 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:55:51.600609 (+ 397us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:55:51.600887 (+ 278us) server_negotiation.cc:1035] Waiting for connection context
0809 22:55:51.601265 (+ 378us) server_negotiation.cc:299] Negotiation successful
0809 22:55:51.601479 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":388}
I20260809 22:55:51.602438 5703 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:55:51.691964 15814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskItYPJR/build/asan/bin/kudu with pid 5505
I20260809 22:55:51.717160 15814 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskItYPJR/build/asan/bin/kudu pid 5716
Aug 09 22:55:51 dist-test-slave-ncp9 krb5kdc[5135](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316148, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.113.131@KRBTEST.COM
I20260809 22:55:51.732702 5944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:55:51.717922 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50080 (local address 127.15.113.131:36817)
0809 22:55:51.718120 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0809 22:55:51.718127 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:55:51.718239 (+ 112us) server_negotiation.cc:407] Connection header received
0809 22:55:51.718421 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:55:51.718425 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:55:51.718505 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:55:51.718594 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:55:51.719925 (+ 1331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:55:51.720757 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:55:51.722220 (+ 1463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:55:51.722697 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:55:51.722809 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:55:51.726932 (+ 4123us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:55:51.726943 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:55:51.726962 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:55:51.727004 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:55:51.729875 (+ 2871us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:55:51.730668 (+ 793us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:55:51.730676 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:55:51.730678 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:55:51.730846 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:55:51.731213 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:55:51.731217 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:55:51.731219 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:55:51.731557 (+ 338us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:55:51.731885 (+ 328us) server_negotiation.cc:1035] Waiting for connection context
0809 22:55:51.732119 (+ 234us) server_negotiation.cc:299] Negotiation successful
0809 22:55:51.732332 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260809 22:55:51.733430 5909 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:55:51.860044 15814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskItYPJR/build/asan/bin/kudu with pid 5716
I20260809 22:55:51.884099 15814 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskItYPJR/build/asan/bin/kudu pid 5157
I20260809 22:55:51.885192 5240 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:55:51.994887 15814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskItYPJR/build/asan/bin/kudu with pid 5157
I20260809 22:55:52.022009 15814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskItYPJR/build/asan/bin/kudu
/tmp/dist-test-taskItYPJR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315935617919-15814-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315935617919-15814-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315935617919-15814-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315935617919-15814-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.113.190:46137
--webserver_interface=127.15.113.190
--webserver_port=45615
--keytab_file=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.190.keytab
--principal=kudu/127.15.113.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.113.148:34295
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.113.190:46137
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315935617919-15814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315935617919-15814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315935617919-15814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:55:52.349277 5996 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:55:52.349539 5996 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:55:52.349637 5996 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:55:52.356524 5996 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 22:55:52.356618 5996 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:55:52.356670 5996 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:55:52.356734 5996 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 22:55:52.356777 5996 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 22:55:52.366073 5996 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.113.148:34295
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315935617919-15814-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315935617919-15814-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.15.113.190:46137
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.190.keytab
--principal=kudu/127.15.113.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.113.190:46137
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315935617919-15814-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.113.190
--webserver_port=45615
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315935617919-15814-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:55:52.367216 5996 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:55:52.368441 5996 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:55:52.392809 6005 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:55:52.393503 6010 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:55:52.394073 5996 server_base.cc:1034] running on GCE node
W20260809 22:55:52.394193 6008 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:55:52.395160 5996 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:55:52.396696 5996 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:55:52.397904 5996 hybrid_clock.cc:648] HybridClock initialized: now 1786316152397823 us; error 65 us; skew 500 ppm
I20260809 22:55:52.398312 5996 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:55:52 dist-test-slave-ncp9 krb5kdc[5135](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316152, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.113.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:55:52.401782 5996 init.cc:377] Logged in from keytab as kudu/127.15.113.190@KRBTEST.COM (short username kudu)
I20260809 22:55:52.403362 5996 webserver.cc:460] Webserver started at http://127.15.113.190:45615/ using document root <none> and password file <none>
I20260809 22:55:52.404090 5996 fs_manager.cc:359] Metadata directory not provided
I20260809 22:55:52.404237 5996 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:55:52.409970 5996 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.001s
I20260809 22:55:52.412774 6020 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:55:52.414069 5996 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 22:55:52.414244 5996 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315935617919-15814-0/minicluster-data/master-0/data,/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315935617919-15814-0/minicluster-data/master-0/wal
uuid: "60c579fec9964c88b77f838d9c13df36"
format_stamp: "Formatted at 2026-08-09 22:55:49 on dist-test-slave-ncp9"
server_key: "b1fa6de1da0146b96208386a306420e6"
server_key_iv: "1740c3981367fca234f4799be5a4c375"
server_key_version: "encryptionkey@0"
I20260809 22:55:52.415282 5996 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315935617919-15814-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315935617919-15814-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315935617919-15814-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:55:52.429323 5996 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:55:52.433499 5996 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:55:52.433744 5996 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:55:52.451684 5996 rpc_server.cc:307] RPC server started. Bound to: 127.15.113.190:46137
I20260809 22:55:52.451694 6104 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.113.190:46137 every 8 connection(s)
I20260809 22:55:52.453214 5996 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315935617919-15814-0/minicluster-data/master-0/data/info.pb
I20260809 22:55:52.455878 15814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskItYPJR/build/asan/bin/kudu as pid 5996
I20260809 22:55:52.456835 15814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskItYPJR/build/asan/bin/kudu
/tmp/dist-test-taskItYPJR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315935617919-15814-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315935617919-15814-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315935617919-15814-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315935617919-15814-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.113.129:34939
--local_ip_for_outbound_sockets=127.15.113.129
--tserver_master_addrs=127.15.113.190:46137
--webserver_port=45525
--webserver_interface=127.15.113.129
--keytab_file=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.129.keytab
--principal=kudu/127.15.113.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.113.148:34295
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315935617919-15814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315935617919-15814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315935617919-15814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 22:55:52.461874 6105 sys_catalog.cc:263] Verifying existing consensus state
I20260809 22:55:52.466250 6105 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 60c579fec9964c88b77f838d9c13df36: Bootstrap starting.
I20260809 22:55:52.485929 6105 log.cc:824] T 00000000000000000000000000000000 P 60c579fec9964c88b77f838d9c13df36: Log is configured to *not* fsync() on all Append() calls
I20260809 22:55:52.497887 6105 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 60c579fec9964c88b77f838d9c13df36: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 22:55:52.498363 6105 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 60c579fec9964c88b77f838d9c13df36: Bootstrap complete.
I20260809 22:55:52.507138 6105 raft_consensus.cc:348] T 00000000000000000000000000000000 P 60c579fec9964c88b77f838d9c13df36 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60c579fec9964c88b77f838d9c13df36" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 46137 } }
I20260809 22:55:52.507952 6105 raft_consensus.cc:729] T 00000000000000000000000000000000 P 60c579fec9964c88b77f838d9c13df36 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60c579fec9964c88b77f838d9c13df36, State: Initialized, Role: FOLLOWER
I20260809 22:55:52.508512 6105 consensus_queue.cc:260] T 00000000000000000000000000000000 P 60c579fec9964c88b77f838d9c13df36 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60c579fec9964c88b77f838d9c13df36" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 46137 } }
I20260809 22:55:52.508742 6105 raft_consensus.cc:388] T 00000000000000000000000000000000 P 60c579fec9964c88b77f838d9c13df36 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 22:55:52.508854 6105 raft_consensus.cc:482] T 00000000000000000000000000000000 P 60c579fec9964c88b77f838d9c13df36 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 22:55:52.509063 6105 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 60c579fec9964c88b77f838d9c13df36 [term 1 FOLLOWER]: Advancing to term 2
I20260809 22:55:52.512674 6105 raft_consensus.cc:504] T 00000000000000000000000000000000 P 60c579fec9964c88b77f838d9c13df36 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60c579fec9964c88b77f838d9c13df36" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 46137 } }
I20260809 22:55:52.513197 6105 leader_election.cc:302] T 00000000000000000000000000000000 P 60c579fec9964c88b77f838d9c13df36 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 60c579fec9964c88b77f838d9c13df36; no voters:
I20260809 22:55:52.513785 6105 leader_election.cc:288] T 00000000000000000000000000000000 P 60c579fec9964c88b77f838d9c13df36 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260809 22:55:52.513996 6111 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 60c579fec9964c88b77f838d9c13df36 [term 2 FOLLOWER]: Leader election won for term 2
I20260809 22:55:52.514933 6111 raft_consensus.cc:686] T 00000000000000000000000000000000 P 60c579fec9964c88b77f838d9c13df36 [term 2 LEADER]: Becoming Leader. State: Replica: 60c579fec9964c88b77f838d9c13df36, State: Running, Role: LEADER
I20260809 22:55:52.515578 6111 consensus_queue.cc:237] T 00000000000000000000000000000000 P 60c579fec9964c88b77f838d9c13df36 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60c579fec9964c88b77f838d9c13df36" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 46137 } }
I20260809 22:55:52.515784 6105 sys_catalog.cc:564] T 00000000000000000000000000000000 P 60c579fec9964c88b77f838d9c13df36 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 22:55:52.518625 6113 sys_catalog.cc:455] T 00000000000000000000000000000000 P 60c579fec9964c88b77f838d9c13df36 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 60c579fec9964c88b77f838d9c13df36. Latest consensus state: current_term: 2 leader_uuid: "60c579fec9964c88b77f838d9c13df36" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60c579fec9964c88b77f838d9c13df36" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 46137 } } }
I20260809 22:55:52.519029 6113 sys_catalog.cc:458] T 00000000000000000000000000000000 P 60c579fec9964c88b77f838d9c13df36 [sys.catalog]: This master's current role is: LEADER
I20260809 22:55:52.519524 6112 sys_catalog.cc:455] T 00000000000000000000000000000000 P 60c579fec9964c88b77f838d9c13df36 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "60c579fec9964c88b77f838d9c13df36" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60c579fec9964c88b77f838d9c13df36" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 46137 } } }
I20260809 22:55:52.519795 6112 sys_catalog.cc:458] T 00000000000000000000000000000000 P 60c579fec9964c88b77f838d9c13df36 [sys.catalog]: This master's current role is: LEADER
I20260809 22:55:52.526734 6130 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 22:55:52.543525 6130 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 22:55:52.545087 6130 catalog_manager.cc:1259] Loaded cluster ID: 85ec813c10da49099b9340d15162601d
I20260809 22:55:52.545253 6130 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 22:55:52.548647 6130 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260809 22:55:52.560420 6146 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260809 22:55:52.564864 6130 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260809 22:55:52.571642 6130 catalog_manager.cc:1504] Loading token signing keys...
I20260809 22:55:52.572835 6130 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 60c579fec9964c88b77f838d9c13df36: Loaded TSK: 0
I20260809 22:55:52.573886 6130 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260809 22:55:52.816475 6109 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:55:52.816745 6109 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:55:52.816931 6109 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:55:52.824194 6109 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:55:52.824304 6109 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:55:52.824452 6109 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.113.129
I20260809 22:55:52.834084 6109 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.113.148:34295
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315935617919-15814-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315935617919-15814-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.129.keytab
--principal=kudu/127.15.113.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.113.129:34939
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315935617919-15814-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.113.129
--webserver_port=45525
--tserver_master_addrs=127.15.113.190:46137
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.113.129
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315935617919-15814-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:55:52.835441 6109 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:55:52.836797 6109 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:55:52.849748 6154 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:55:52.849781 6153 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:55:52.850435 6157 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:55:52.850813 6109 server_base.cc:1034] running on GCE node
I20260809 22:55:52.851362 6109 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:55:52.852288 6109 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:55:52.853492 6109 hybrid_clock.cc:648] HybridClock initialized: now 1786316152853441 us; error 45 us; skew 500 ppm
I20260809 22:55:52.853785 6109 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:55:52 dist-test-slave-ncp9 krb5kdc[5135](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316152, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.113.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:55:52.870560 6109 init.cc:377] Logged in from keytab as kudu/127.15.113.129@KRBTEST.COM (short username kudu)
I20260809 22:55:52.872274 6109 webserver.cc:460] Webserver started at http://127.15.113.129:45525/ using document root <none> and password file <none>
I20260809 22:55:52.873071 6109 fs_manager.cc:359] Metadata directory not provided
I20260809 22:55:52.873229 6109 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:55:52.878898 6109 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260809 22:55:52.881752 6170 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:55:52.883164 6109 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 22:55:52.883324 6109 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315935617919-15814-0/minicluster-data/ts-0/data,/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315935617919-15814-0/minicluster-data/ts-0/wal
uuid: "e9491fd35ae64be7be6fa206b46d187a"
format_stamp: "Formatted at 2026-08-09 22:55:49 on dist-test-slave-ncp9"
server_key: "688d8c1cc62f3dd447fd01428a4507b3"
server_key_iv: "1f230b0c43cfd79a3fc22fe895743930"
server_key_version: "encryptionkey@0"
I20260809 22:55:52.884351 6109 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315935617919-15814-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315935617919-15814-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315935617919-15814-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:55:52.911119 6109 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:55:52.914989 6109 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:55:52.915237 6109 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:55:52.916220 6109 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:55:52.917565 6109 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:55:52.917659 6109 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:55:52.917738 6109 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:55:52.917793 6109 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:55:52.947692 6109 rpc_server.cc:307] RPC server started. Bound to: 127.15.113.129:34939
I20260809 22:55:52.947754 6348 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.113.129:34939 every 8 connection(s)
I20260809 22:55:52.949081 6109 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315935617919-15814-0/minicluster-data/ts-0/data/info.pb
I20260809 22:55:52.956459 15814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskItYPJR/build/asan/bin/kudu as pid 6109
I20260809 22:55:52.957291 15814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskItYPJR/build/asan/bin/kudu
/tmp/dist-test-taskItYPJR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315935617919-15814-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315935617919-15814-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315935617919-15814-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315935617919-15814-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.113.130:39003
--local_ip_for_outbound_sockets=127.15.113.130
--tserver_master_addrs=127.15.113.190:46137
--webserver_port=40721
--webserver_interface=127.15.113.130
--keytab_file=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.130.keytab
--principal=kudu/127.15.113.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.113.148:34295
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315935617919-15814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315935617919-15814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315935617919-15814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 22:55:52 dist-test-slave-ncp9 krb5kdc[5135](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316152, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.113.129@KRBTEST.COM for kudu/127.15.113.190@KRBTEST.COM
I20260809 22:55:52.971531 6358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:55:52.952149 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:46137 (local address 127.15.113.129:54301)
0809 22:55:52.952989 (+ 840us) negotiation.cc:107] Waiting for socket to connect
0809 22:55:52.953035 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0809 22:55:52.954743 (+ 1708us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:55:52.955958 (+ 1215us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:55:52.955969 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:55:52.956682 (+ 713us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:55:52.957486 (+ 804us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:55:52.957513 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:55:52.959057 (+ 1544us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:55:52.959062 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:55:52.960750 (+ 1688us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:55:52.960765 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:55:52.961748 (+ 983us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:55:52.961757 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:55:52.962008 (+ 251us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:55:52.962607 (+ 599us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:55:52.962628 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:55:52.964672 (+ 2044us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:55:52.967787 (+ 3115us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:55:52.967796 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:55:52.967809 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:55:52.968207 (+ 398us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:55:52.968681 (+ 474us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:55:52.968684 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:55:52.968687 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:55:52.968870 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:55:52.969710 (+ 840us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:55:52.969716 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:55:52.969989 (+ 273us) client_negotiation.cc:769] Sending connection context
0809 22:55:52.970200 (+ 211us) client_negotiation.cc:240] Negotiation successful
0809 22:55:52.970558 (+ 358us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":524,"thread_start_us":311,"threads_started":1}
I20260809 22:55:52.971966 6356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:55:52.951852 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.113.129:54301 (local address 127.15.113.190:46137)
0809 22:55:52.952376 (+ 524us) server_negotiation.cc:206] Beginning negotiation
0809 22:55:52.952387 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 22:55:52.953144 (+ 757us) server_negotiation.cc:407] Connection header received
0809 22:55:52.955184 (+ 2040us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:55:52.955197 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:55:52.955508 (+ 311us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:55:52.955803 (+ 295us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:55:52.957709 (+ 1906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:55:52.958895 (+ 1186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:55:52.960868 (+ 1973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:55:52.961529 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:55:52.961713 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:55:52.964936 (+ 3223us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:55:52.964947 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:55:52.964960 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:55:52.964993 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:55:52.967657 (+ 2664us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:55:52.968333 (+ 676us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:55:52.968341 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:55:52.968348 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:55:52.968558 (+ 210us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:55:52.968991 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:55:52.968996 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:55:52.968998 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:55:52.969584 (+ 586us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:55:52.969784 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0809 22:55:52.970465 (+ 681us) server_negotiation.cc:299] Negotiation successful
0809 22:55:52.970858 (+ 393us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":345,"thread_start_us":230,"threads_started":1}
I20260809 22:55:52.974546 6351 heartbeater.cc:347] Connected to a master server at 127.15.113.190:46137
I20260809 22:55:52.974885 6351 heartbeater.cc:464] Registering TS with master...
I20260809 22:55:52.975692 6351 heartbeater.cc:511] Master 127.15.113.190:46137 requested a full tablet report, sending...
I20260809 22:55:52.977876 6051 ts_manager.cc:194] Registered new tserver with Master: e9491fd35ae64be7be6fa206b46d187a (127.15.113.129:34939)
I20260809 22:55:52.979342 6051 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.113.129@KRBTEST.COM'} at 127.15.113.129:54301
W20260809 22:55:53.285116 6359 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:55:53.285389 6359 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:55:53.285549 6359 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:55:53.292578 6359 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:55:53.292685 6359 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:55:53.292831 6359 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.113.130
I20260809 22:55:53.302032 6359 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.113.148:34295
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315935617919-15814-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315935617919-15814-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.130.keytab
--principal=kudu/127.15.113.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.113.130:39003
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315935617919-15814-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.113.130
--webserver_port=40721
--tserver_master_addrs=127.15.113.190:46137
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.113.130
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315935617919-15814-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:55:53.303225 6359 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:55:53.304473 6359 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:55:53.316861 6364 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:55:53.316944 6365 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:55:53.317065 6367 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:55:53.317940 6359 server_base.cc:1034] running on GCE node
I20260809 22:55:53.318485 6359 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:55:53.319339 6359 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:55:53.320559 6359 hybrid_clock.cc:648] HybridClock initialized: now 1786316153320503 us; error 51 us; skew 500 ppm
I20260809 22:55:53.320822 6359 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:55:53 dist-test-slave-ncp9 krb5kdc[5135](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316153, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.113.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:55:53.323818 6359 init.cc:377] Logged in from keytab as kudu/127.15.113.130@KRBTEST.COM (short username kudu)
I20260809 22:55:53.325239 6359 webserver.cc:460] Webserver started at http://127.15.113.130:40721/ using document root <none> and password file <none>
I20260809 22:55:53.325881 6359 fs_manager.cc:359] Metadata directory not provided
I20260809 22:55:53.326012 6359 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:55:53.331086 6359 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.002s
I20260809 22:55:53.333639 6383 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:55:53.334909 6359 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 22:55:53.335043 6359 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315935617919-15814-0/minicluster-data/ts-1/data,/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315935617919-15814-0/minicluster-data/ts-1/wal
uuid: "6fb2492799864fb8b944c12e0aa53c0a"
format_stamp: "Formatted at 2026-08-09 22:55:50 on dist-test-slave-ncp9"
server_key: "3bd00e7d274a6999ef76a1d8237a5ab7"
server_key_iv: "29b30cacc051bf72a62afb639737ac6c"
server_key_version: "encryptionkey@0"
I20260809 22:55:53.335956 6359 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315935617919-15814-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315935617919-15814-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315935617919-15814-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:55:53.347380 6359 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:55:53.351292 6359 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:55:53.351506 6359 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:55:53.352345 6359 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:55:53.353596 6359 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:55:53.353681 6359 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:55:53.353760 6359 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:55:53.353814 6359 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:55:53.382467 6359 rpc_server.cc:307] RPC server started. Bound to: 127.15.113.130:39003
I20260809 22:55:53.382529 6565 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.113.130:39003 every 8 connection(s)
I20260809 22:55:53.383836 6359 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315935617919-15814-0/minicluster-data/ts-1/data/info.pb
I20260809 22:55:53.385231 15814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskItYPJR/build/asan/bin/kudu as pid 6359
I20260809 22:55:53.385854 15814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskItYPJR/build/asan/bin/kudu
/tmp/dist-test-taskItYPJR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315935617919-15814-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315935617919-15814-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315935617919-15814-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315935617919-15814-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.113.131:36817
--local_ip_for_outbound_sockets=127.15.113.131
--tserver_master_addrs=127.15.113.190:46137
--webserver_port=41869
--webserver_interface=127.15.113.131
--keytab_file=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.131.keytab
--principal=kudu/127.15.113.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.113.148:34295
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315935617919-15814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315935617919-15814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315935617919-15814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 22:55:53 dist-test-slave-ncp9 krb5kdc[5135](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316153, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.113.130@KRBTEST.COM for kudu/127.15.113.190@KRBTEST.COM
I20260809 22:55:53.405720 6356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:55:53.387222 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.113.130:33555 (local address 127.15.113.190:46137)
0809 22:55:53.387410 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0809 22:55:53.387415 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:55:53.387778 (+ 363us) server_negotiation.cc:407] Connection header received
0809 22:55:53.390130 (+ 2352us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:55:53.390135 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:55:53.390219 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:55:53.390311 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:55:53.392217 (+ 1906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:55:53.393097 (+ 880us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:55:53.395189 (+ 2092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:55:53.395769 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:55:53.395864 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:55:53.399928 (+ 4064us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:55:53.399935 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:55:53.399941 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:55:53.399981 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:55:53.402880 (+ 2899us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:55:53.403532 (+ 652us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:55:53.403537 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:55:53.403540 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:55:53.403695 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:55:53.404055 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:55:53.404059 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:55:53.404062 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:55:53.404375 (+ 313us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:55:53.404516 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0809 22:55:53.405165 (+ 649us) server_negotiation.cc:299] Negotiation successful
0809 22:55:53.405378 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260809 22:55:53.406404 6571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:55:53.386722 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:46137 (local address 127.15.113.130:33555)
0809 22:55:53.387582 (+ 860us) negotiation.cc:107] Waiting for socket to connect
0809 22:55:53.387649 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0809 22:55:53.389843 (+ 2194us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:55:53.390571 (+ 728us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:55:53.390582 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:55:53.391258 (+ 676us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:55:53.392044 (+ 786us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:55:53.392069 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:55:53.393270 (+ 1201us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:55:53.393275 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:55:53.395002 (+ 1727us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:55:53.395018 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:55:53.396186 (+ 1168us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:55:53.396193 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:55:53.396453 (+ 260us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:55:53.397196 (+ 743us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:55:53.397224 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:55:53.399713 (+ 2489us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:55:53.403007 (+ 3294us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:55:53.403013 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:55:53.403024 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:55:53.403415 (+ 391us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:55:53.403789 (+ 374us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:55:53.403791 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:55:53.403794 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:55:53.403958 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:55:53.404489 (+ 531us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:55:53.404497 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:55:53.404789 (+ 292us) client_negotiation.cc:769] Sending connection context
0809 22:55:53.405038 (+ 249us) client_negotiation.cc:240] Negotiation successful
0809 22:55:53.405395 (+ 357us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":492,"thread_start_us":287,"threads_started":1}
I20260809 22:55:53.407758 6568 heartbeater.cc:347] Connected to a master server at 127.15.113.190:46137
I20260809 22:55:53.408041 6568 heartbeater.cc:464] Registering TS with master...
I20260809 22:55:53.408716 6568 heartbeater.cc:511] Master 127.15.113.190:46137 requested a full tablet report, sending...
I20260809 22:55:53.410110 6051 ts_manager.cc:194] Registered new tserver with Master: 6fb2492799864fb8b944c12e0aa53c0a (127.15.113.130:39003)
I20260809 22:55:53.410951 6051 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.113.130@KRBTEST.COM'} at 127.15.113.130:33555
W20260809 22:55:53.730883 6572 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:55:53.731172 6572 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:55:53.731315 6572 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:55:53.738370 6572 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:55:53.738480 6572 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:55:53.738632 6572 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.113.131
I20260809 22:55:53.748175 6572 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.113.148:34295
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315935617919-15814-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315935617919-15814-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.131.keytab
--principal=kudu/127.15.113.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.113.131:36817
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315935617919-15814-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.113.131
--webserver_port=41869
--tserver_master_addrs=127.15.113.190:46137
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.113.131
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315935617919-15814-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:55:53.749426 6572 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:55:53.750733 6572 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:55:53.762941 6582 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:55:53.763019 6579 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:55:53.762991 6580 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:55:53.763787 6572 server_base.cc:1034] running on GCE node
I20260809 22:55:53.764354 6572 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:55:53.765229 6572 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:55:53.766417 6572 hybrid_clock.cc:648] HybridClock initialized: now 1786316153766371 us; error 36 us; skew 500 ppm
I20260809 22:55:53.766691 6572 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:55:53 dist-test-slave-ncp9 krb5kdc[5135](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316153, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.113.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:55:53.769619 6572 init.cc:377] Logged in from keytab as kudu/127.15.113.131@KRBTEST.COM (short username kudu)
I20260809 22:55:53.770879 6572 webserver.cc:460] Webserver started at http://127.15.113.131:41869/ using document root <none> and password file <none>
I20260809 22:55:53.771500 6572 fs_manager.cc:359] Metadata directory not provided
I20260809 22:55:53.771625 6572 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:55:53.776557 6572 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.001s
I20260809 22:55:53.779081 6592 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:55:53.780256 6572 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 22:55:53.780395 6572 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315935617919-15814-0/minicluster-data/ts-2/data,/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315935617919-15814-0/minicluster-data/ts-2/wal
uuid: "82971d28843442a7a9c8338a4ad220bb"
format_stamp: "Formatted at 2026-08-09 22:55:50 on dist-test-slave-ncp9"
server_key: "159f3a218e0536600f31a263f5e25b04"
server_key_iv: "0359d24e8ea6f74a0f00667a35524811"
server_key_version: "encryptionkey@0"
I20260809 22:55:53.781345 6572 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315935617919-15814-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315935617919-15814-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315935617919-15814-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:55:53.793934 6572 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:55:53.797333 6572 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:55:53.797559 6572 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:55:53.798393 6572 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:55:53.799600 6572 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:55:53.799693 6572 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:55:53.799791 6572 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:55:53.799860 6572 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:55:53.829629 6572 rpc_server.cc:307] RPC server started. Bound to: 127.15.113.131:36817
I20260809 22:55:53.829694 6763 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.113.131:36817 every 8 connection(s)
I20260809 22:55:53.831108 6572 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315935617919-15814-0/minicluster-data/ts-2/data/info.pb
I20260809 22:55:53.838685 15814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskItYPJR/build/asan/bin/kudu as pid 6572
Aug 09 22:55:53 dist-test-slave-ncp9 krb5kdc[5135](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316153, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.113.131@KRBTEST.COM for kudu/127.15.113.190@KRBTEST.COM
I20260809 22:55:53.850955 6356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:55:53.833545 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.113.131:51201 (local address 127.15.113.190:46137)
0809 22:55:53.833771 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0809 22:55:53.833777 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:55:53.834905 (+ 1128us) server_negotiation.cc:407] Connection header received
0809 22:55:53.836609 (+ 1704us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:55:53.836614 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:55:53.836687 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:55:53.836771 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:55:53.838172 (+ 1401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:55:53.839181 (+ 1009us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:55:53.841369 (+ 2188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:55:53.841790 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:55:53.841899 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:55:53.845258 (+ 3359us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:55:53.845264 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:55:53.845269 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:55:53.845308 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:55:53.848026 (+ 2718us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:55:53.848683 (+ 657us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:55:53.848688 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:55:53.848692 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:55:53.848857 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:55:53.849278 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:55:53.849284 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:55:53.849287 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:55:53.849617 (+ 330us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:55:53.849793 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0809 22:55:53.850393 (+ 600us) server_negotiation.cc:299] Negotiation successful
0809 22:55:53.850634 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260809 22:55:53.851755 6767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:55:53.834062 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:46137 (local address 127.15.113.131:51201)
0809 22:55:53.834748 (+ 686us) negotiation.cc:107] Waiting for socket to connect
0809 22:55:53.834794 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0809 22:55:53.836374 (+ 1580us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:55:53.836993 (+ 619us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:55:53.837001 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:55:53.837475 (+ 474us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:55:53.838026 (+ 551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:55:53.838042 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:55:53.839368 (+ 1326us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:55:53.839373 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:55:53.841183 (+ 1810us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:55:53.841206 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:55:53.842006 (+ 800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:55:53.842013 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:55:53.842244 (+ 231us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:55:53.842837 (+ 593us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:55:53.842858 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:55:53.845068 (+ 2210us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:55:53.848147 (+ 3079us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:55:53.848153 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:55:53.848164 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:55:53.848551 (+ 387us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:55:53.848985 (+ 434us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:55:53.848988 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:55:53.848990 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:55:53.849160 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:55:53.849753 (+ 593us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:55:53.849759 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:55:53.850030 (+ 271us) client_negotiation.cc:769] Sending connection context
0809 22:55:53.850239 (+ 209us) client_negotiation.cc:240] Negotiation successful
0809 22:55:53.850575 (+ 336us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":444,"thread_start_us":260,"threads_started":1}
I20260809 22:55:53.853231 6764 heartbeater.cc:347] Connected to a master server at 127.15.113.190:46137
I20260809 22:55:53.853503 6764 heartbeater.cc:464] Registering TS with master...
I20260809 22:55:53.853518 6768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:55:53.839654 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58082 (local address 127.15.113.190:46137)
0809 22:55:53.840109 (+ 455us) server_negotiation.cc:206] Beginning negotiation
0809 22:55:53.840115 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:55:53.840419 (+ 304us) server_negotiation.cc:407] Connection header received
0809 22:55:53.840593 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:55:53.840598 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:55:53.840658 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:55:53.840745 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:55:53.842333 (+ 1588us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:55:53.843046 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:55:53.844663 (+ 1617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:55:53.845146 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:55:53.845268 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:55:53.847694 (+ 2426us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:55:53.847700 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:55:53.847705 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:55:53.847744 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:55:53.850788 (+ 3044us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:55:53.851572 (+ 784us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:55:53.851576 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:55:53.851578 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:55:53.851691 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:55:53.852100 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:55:53.852105 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:55:53.852108 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:55:53.852438 (+ 330us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:55:53.852647 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0809 22:55:53.852964 (+ 317us) server_negotiation.cc:299] Negotiation successful
0809 22:55:53.853184 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":337,"spinlock_wait_cycles":896,"thread_start_us":217,"threads_started":1}
I20260809 22:55:53.854184 6764 heartbeater.cc:511] Master 127.15.113.190:46137 requested a full tablet report, sending...
I20260809 22:55:53.855641 6048 ts_manager.cc:194] Registered new tserver with Master: 82971d28843442a7a9c8338a4ad220bb (127.15.113.131:36817)
I20260809 22:55:53.856519 6048 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.113.131@KRBTEST.COM'} at 127.15.113.131:51201
I20260809 22:55:53.867755 15814 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 22:55:53.874215 6768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:55:53.868555 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58084 (local address 127.15.113.190:46137)
0809 22:55:53.868685 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0809 22:55:53.868689 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 22:55:53.869166 (+ 477us) server_negotiation.cc:407] Connection header received
0809 22:55:53.869368 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:55:53.869371 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:55:53.869503 (+ 132us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:55:53.869625 (+ 122us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 22:55:53.870047 (+ 422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:55:53.870703 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:55:53.872156 (+ 1453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:55:53.872568 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:55:53.872674 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:55:53.873227 (+ 553us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 22:55:53.873652 (+ 425us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 22:55:53.873790 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0809 22:55:53.873890 (+ 100us) server_negotiation.cc:299] Negotiation successful
0809 22:55:53.873982 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260809 22:55:53.875010 6048 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:58084:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 22:55:53.882047 6048 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 22:55:53.902709 6780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:55:53.893034 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.130:39003 (local address 127.0.0.1:49240)
0809 22:55:53.894609 (+ 1575us) negotiation.cc:107] Waiting for socket to connect
0809 22:55:53.894671 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0809 22:55:53.894850 (+ 179us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:55:53.895550 (+ 700us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:55:53.895557 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:55:53.895582 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:55:53.896023 (+ 441us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:55:53.896037 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:55:53.898305 (+ 2268us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:55:53.898310 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:55:53.900394 (+ 2084us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:55:53.900409 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:55:53.901773 (+ 1364us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:55:53.901781 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:55:53.901973 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:55:53.902031 (+ 58us) client_negotiation.cc:769] Sending connection context
0809 22:55:53.902162 (+ 131us) client_negotiation.cc:240] Negotiation successful
0809 22:55:53.902350 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1336,"thread_start_us":3204,"threads_started":1}
I20260809 22:55:53.903357 6781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:55:53.893175 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49240 (local address 127.15.113.130:39003)
0809 22:55:53.894099 (+ 924us) server_negotiation.cc:206] Beginning negotiation
0809 22:55:53.894106 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:55:53.894787 (+ 681us) server_negotiation.cc:407] Connection header received
0809 22:55:53.895055 (+ 268us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:55:53.895064 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:55:53.895327 (+ 263us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:55:53.896336 (+ 1009us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:55:53.896632 (+ 296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:55:53.898098 (+ 1466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:55:53.900546 (+ 2448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:55:53.901536 (+ 990us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:55:53.901719 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:55:53.901895 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0809 22:55:53.902858 (+ 963us) server_negotiation.cc:299] Negotiation successful
0809 22:55:53.903021 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":781,"thread_start_us":644,"threads_started":1}
I20260809 22:55:53.904143 6782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:55:53.893112 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.129:34939 (local address 127.0.0.1:57716)
0809 22:55:53.895359 (+ 2247us) negotiation.cc:107] Waiting for socket to connect
0809 22:55:53.895375 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 22:55:53.895481 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:55:53.897332 (+ 1851us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:55:53.897336 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:55:53.897354 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:55:53.897913 (+ 559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:55:53.897926 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:55:53.899787 (+ 1861us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:55:53.899793 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:55:53.901701 (+ 1908us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:55:53.901716 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:55:53.903642 (+ 1926us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:55:53.903646 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:55:53.903773 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:55:53.903795 (+ 22us) client_negotiation.cc:769] Sending connection context
0809 22:55:53.903847 (+ 52us) client_negotiation.cc:240] Negotiation successful
0809 22:55:53.903925 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2137,"thread_start_us":1125,"threads_started":1}
I20260809 22:55:53.905189 6783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:55:53.893617 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57716 (local address 127.15.113.129:34939)
0809 22:55:53.896837 (+ 3220us) server_negotiation.cc:206] Beginning negotiation
0809 22:55:53.896843 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:55:53.896865 (+ 22us) server_negotiation.cc:407] Connection header received
0809 22:55:53.896978 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:55:53.896985 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:55:53.897152 (+ 167us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:55:53.897603 (+ 451us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:55:53.898117 (+ 514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:55:53.899642 (+ 1525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:55:53.901985 (+ 2343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:55:53.903416 (+ 1431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:55:53.904462 (+ 1046us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:55:53.904615 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0809 22:55:53.904693 (+ 78us) server_negotiation.cc:299] Negotiation successful
0809 22:55:53.904850 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3082,"thread_start_us":2951,"threads_started":1}
I20260809 22:55:53.905391 6779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:55:53.893584 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.131:36817 (local address 127.0.0.1:50082)
0809 22:55:53.896502 (+ 2918us) negotiation.cc:107] Waiting for socket to connect
0809 22:55:53.896519 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 22:55:53.896627 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:55:53.897295 (+ 668us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:55:53.897300 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:55:53.897326 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:55:53.897798 (+ 472us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:55:53.897811 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:55:53.900990 (+ 3179us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:55:53.900995 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:55:53.902937 (+ 1942us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:55:53.902951 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:55:53.904646 (+ 1695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:55:53.904653 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:55:53.904823 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:55:53.904853 (+ 30us) client_negotiation.cc:769] Sending connection context
0809 22:55:53.904969 (+ 116us) client_negotiation.cc:240] Negotiation successful
0809 22:55:53.905087 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2815,"thread_start_us":1545,"threads_started":1}
I20260809 22:55:53.905471 6788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:55:53.894092 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50082 (local address 127.15.113.131:36817)
0809 22:55:53.894670 (+ 578us) server_negotiation.cc:206] Beginning negotiation
0809 22:55:53.894678 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 22:55:53.896755 (+ 2077us) server_negotiation.cc:407] Connection header received
0809 22:55:53.896851 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:55:53.896858 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:55:53.897103 (+ 245us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:55:53.897278 (+ 175us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:55:53.897980 (+ 702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:55:53.899658 (+ 1678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:55:53.903112 (+ 3454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:55:53.904428 (+ 1316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:55:53.904614 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:55:53.904780 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0809 22:55:53.905009 (+ 229us) server_negotiation.cc:299] Negotiation successful
0809 22:55:53.905176 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":418,"thread_start_us":267,"threads_started":1}
I20260809 22:55:53.907388 6450 tablet_service.cc:1467] Processing CreateTablet for tablet b274f20c33db4e75bad54d9e65792cd7 (DEFAULT_TABLE table=test-table [id=a7c0b42e69ea4ca281bbe21670b61a6d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:55:53.908496 6450 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b274f20c33db4e75bad54d9e65792cd7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:55:53.908738 6247 tablet_service.cc:1467] Processing CreateTablet for tablet b274f20c33db4e75bad54d9e65792cd7 (DEFAULT_TABLE table=test-table [id=a7c0b42e69ea4ca281bbe21670b61a6d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:55:53.908735 6665 tablet_service.cc:1467] Processing CreateTablet for tablet b274f20c33db4e75bad54d9e65792cd7 (DEFAULT_TABLE table=test-table [id=a7c0b42e69ea4ca281bbe21670b61a6d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:55:53.910102 6247 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b274f20c33db4e75bad54d9e65792cd7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:55:53.910099 6665 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b274f20c33db4e75bad54d9e65792cd7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:55:53.920506 6789 tablet_bootstrap.cc:492] T b274f20c33db4e75bad54d9e65792cd7 P 6fb2492799864fb8b944c12e0aa53c0a: Bootstrap starting.
I20260809 22:55:53.923385 6789 tablet_bootstrap.cc:654] T b274f20c33db4e75bad54d9e65792cd7 P 6fb2492799864fb8b944c12e0aa53c0a: Neither blocks nor log segments found. Creating new log.
I20260809 22:55:53.924564 6789 log.cc:824] T b274f20c33db4e75bad54d9e65792cd7 P 6fb2492799864fb8b944c12e0aa53c0a: Log is configured to *not* fsync() on all Append() calls
I20260809 22:55:53.925772 6790 tablet_bootstrap.cc:492] T b274f20c33db4e75bad54d9e65792cd7 P e9491fd35ae64be7be6fa206b46d187a: Bootstrap starting.
I20260809 22:55:53.928128 6791 tablet_bootstrap.cc:492] T b274f20c33db4e75bad54d9e65792cd7 P 82971d28843442a7a9c8338a4ad220bb: Bootstrap starting.
I20260809 22:55:53.929279 6790 tablet_bootstrap.cc:654] T b274f20c33db4e75bad54d9e65792cd7 P e9491fd35ae64be7be6fa206b46d187a: Neither blocks nor log segments found. Creating new log.
I20260809 22:55:53.929432 6789 tablet_bootstrap.cc:492] T b274f20c33db4e75bad54d9e65792cd7 P 6fb2492799864fb8b944c12e0aa53c0a: No bootstrap required, opened a new log
I20260809 22:55:53.929724 6789 ts_tablet_manager.cc:1397] T b274f20c33db4e75bad54d9e65792cd7 P 6fb2492799864fb8b944c12e0aa53c0a: Time spent bootstrapping tablet: real 0.009s user 0.008s sys 0.000s
I20260809 22:55:53.930488 6790 log.cc:824] T b274f20c33db4e75bad54d9e65792cd7 P e9491fd35ae64be7be6fa206b46d187a: Log is configured to *not* fsync() on all Append() calls
I20260809 22:55:53.931025 6791 tablet_bootstrap.cc:654] T b274f20c33db4e75bad54d9e65792cd7 P 82971d28843442a7a9c8338a4ad220bb: Neither blocks nor log segments found. Creating new log.
I20260809 22:55:53.932212 6791 log.cc:824] T b274f20c33db4e75bad54d9e65792cd7 P 82971d28843442a7a9c8338a4ad220bb: Log is configured to *not* fsync() on all Append() calls
I20260809 22:55:53.933499 6790 tablet_bootstrap.cc:492] T b274f20c33db4e75bad54d9e65792cd7 P e9491fd35ae64be7be6fa206b46d187a: No bootstrap required, opened a new log
I20260809 22:55:53.933815 6790 ts_tablet_manager.cc:1397] T b274f20c33db4e75bad54d9e65792cd7 P e9491fd35ae64be7be6fa206b46d187a: Time spent bootstrapping tablet: real 0.008s user 0.008s sys 0.000s
I20260809 22:55:53.935648 6791 tablet_bootstrap.cc:492] T b274f20c33db4e75bad54d9e65792cd7 P 82971d28843442a7a9c8338a4ad220bb: No bootstrap required, opened a new log
I20260809 22:55:53.935866 6791 ts_tablet_manager.cc:1397] T b274f20c33db4e75bad54d9e65792cd7 P 82971d28843442a7a9c8338a4ad220bb: Time spent bootstrapping tablet: real 0.008s user 0.003s sys 0.003s
I20260809 22:55:53.939469 6789 raft_consensus.cc:348] T b274f20c33db4e75bad54d9e65792cd7 P 6fb2492799864fb8b944c12e0aa53c0a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fb2492799864fb8b944c12e0aa53c0a" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 39003 } } peers { permanent_uuid: "e9491fd35ae64be7be6fa206b46d187a" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 34939 } } peers { permanent_uuid: "82971d28843442a7a9c8338a4ad220bb" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 36817 } }
I20260809 22:55:53.939879 6789 raft_consensus.cc:374] T b274f20c33db4e75bad54d9e65792cd7 P 6fb2492799864fb8b944c12e0aa53c0a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:55:53.939975 6789 raft_consensus.cc:729] T b274f20c33db4e75bad54d9e65792cd7 P 6fb2492799864fb8b944c12e0aa53c0a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fb2492799864fb8b944c12e0aa53c0a, State: Initialized, Role: FOLLOWER
I20260809 22:55:53.940686 6789 consensus_queue.cc:260] T b274f20c33db4e75bad54d9e65792cd7 P 6fb2492799864fb8b944c12e0aa53c0a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fb2492799864fb8b944c12e0aa53c0a" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 39003 } } peers { permanent_uuid: "e9491fd35ae64be7be6fa206b46d187a" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 34939 } } peers { permanent_uuid: "82971d28843442a7a9c8338a4ad220bb" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 36817 } }
I20260809 22:55:53.941887 6568 heartbeater.cc:503] Master 127.15.113.190:46137 was elected leader, sending a full tablet report...
I20260809 22:55:53.943823 6791 raft_consensus.cc:348] T b274f20c33db4e75bad54d9e65792cd7 P 82971d28843442a7a9c8338a4ad220bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fb2492799864fb8b944c12e0aa53c0a" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 39003 } } peers { permanent_uuid: "e9491fd35ae64be7be6fa206b46d187a" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 34939 } } peers { permanent_uuid: "82971d28843442a7a9c8338a4ad220bb" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 36817 } }
I20260809 22:55:53.943958 6790 raft_consensus.cc:348] T b274f20c33db4e75bad54d9e65792cd7 P e9491fd35ae64be7be6fa206b46d187a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fb2492799864fb8b944c12e0aa53c0a" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 39003 } } peers { permanent_uuid: "e9491fd35ae64be7be6fa206b46d187a" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 34939 } } peers { permanent_uuid: "82971d28843442a7a9c8338a4ad220bb" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 36817 } }
I20260809 22:55:53.944304 6791 raft_consensus.cc:374] T b274f20c33db4e75bad54d9e65792cd7 P 82971d28843442a7a9c8338a4ad220bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:55:53.944425 6791 raft_consensus.cc:729] T b274f20c33db4e75bad54d9e65792cd7 P 82971d28843442a7a9c8338a4ad220bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 82971d28843442a7a9c8338a4ad220bb, State: Initialized, Role: FOLLOWER
I20260809 22:55:53.944550 6790 raft_consensus.cc:374] T b274f20c33db4e75bad54d9e65792cd7 P e9491fd35ae64be7be6fa206b46d187a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:55:53.944708 6790 raft_consensus.cc:729] T b274f20c33db4e75bad54d9e65792cd7 P e9491fd35ae64be7be6fa206b46d187a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9491fd35ae64be7be6fa206b46d187a, State: Initialized, Role: FOLLOWER
I20260809 22:55:53.945173 6791 consensus_queue.cc:260] T b274f20c33db4e75bad54d9e65792cd7 P 82971d28843442a7a9c8338a4ad220bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fb2492799864fb8b944c12e0aa53c0a" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 39003 } } peers { permanent_uuid: "e9491fd35ae64be7be6fa206b46d187a" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 34939 } } peers { permanent_uuid: "82971d28843442a7a9c8338a4ad220bb" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 36817 } }
I20260809 22:55:53.945451 6790 consensus_queue.cc:260] T b274f20c33db4e75bad54d9e65792cd7 P e9491fd35ae64be7be6fa206b46d187a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fb2492799864fb8b944c12e0aa53c0a" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 39003 } } peers { permanent_uuid: "e9491fd35ae64be7be6fa206b46d187a" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 34939 } } peers { permanent_uuid: "82971d28843442a7a9c8338a4ad220bb" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 36817 } }
I20260809 22:55:53.946530 6789 ts_tablet_manager.cc:1428] T b274f20c33db4e75bad54d9e65792cd7 P 6fb2492799864fb8b944c12e0aa53c0a: Time spent starting tablet: real 0.017s user 0.006s sys 0.007s
I20260809 22:55:53.946605 6798 raft_consensus.cc:482] T b274f20c33db4e75bad54d9e65792cd7 P 6fb2492799864fb8b944c12e0aa53c0a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 22:55:53.947523 6798 raft_consensus.cc:504] T b274f20c33db4e75bad54d9e65792cd7 P 6fb2492799864fb8b944c12e0aa53c0a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fb2492799864fb8b944c12e0aa53c0a" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 39003 } } peers { permanent_uuid: "e9491fd35ae64be7be6fa206b46d187a" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 34939 } } peers { permanent_uuid: "82971d28843442a7a9c8338a4ad220bb" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 36817 } }
I20260809 22:55:53.952328 6764 heartbeater.cc:503] Master 127.15.113.190:46137 was elected leader, sending a full tablet report...
I20260809 22:55:53.953322 6791 ts_tablet_manager.cc:1428] T b274f20c33db4e75bad54d9e65792cd7 P 82971d28843442a7a9c8338a4ad220bb: Time spent starting tablet: real 0.017s user 0.013s sys 0.005s
I20260809 22:55:53.954375 6798 leader_election.cc:288] T b274f20c33db4e75bad54d9e65792cd7 P 6fb2492799864fb8b944c12e0aa53c0a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e9491fd35ae64be7be6fa206b46d187a (127.15.113.129:34939), 82971d28843442a7a9c8338a4ad220bb (127.15.113.131:36817)
I20260809 22:55:53.956115 6351 heartbeater.cc:503] Master 127.15.113.190:46137 was elected leader, sending a full tablet report...
I20260809 22:55:53.956604 6790 ts_tablet_manager.cc:1428] T b274f20c33db4e75bad54d9e65792cd7 P e9491fd35ae64be7be6fa206b46d187a: Time spent starting tablet: real 0.023s user 0.010s sys 0.005s
I20260809 22:55:53.961074 6783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:55:53.951906 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.113.130:37077 (local address 127.15.113.129:34939)
0809 22:55:53.952084 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0809 22:55:53.952091 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:55:53.954153 (+ 2062us) server_negotiation.cc:407] Connection header received
0809 22:55:53.954323 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:55:53.954328 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:55:53.954403 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:55:53.954488 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:55:53.955304 (+ 816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:55:53.956426 (+ 1122us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:55:53.958725 (+ 2299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:55:53.959941 (+ 1216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:55:53.960034 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:55:53.960104 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0809 22:55:53.960660 (+ 556us) server_negotiation.cc:299] Negotiation successful
0809 22:55:53.960808 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260809 22:55:53.961117 6801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:55:53.951463 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.129:34939 (local address 127.15.113.130:37077)
0809 22:55:53.954044 (+ 2581us) negotiation.cc:107] Waiting for socket to connect
0809 22:55:53.954062 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 22:55:53.954194 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:55:53.954655 (+ 461us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:55:53.954666 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:55:53.954708 (+ 42us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:55:53.955115 (+ 407us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:55:53.955138 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:55:53.956538 (+ 1400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:55:53.956543 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:55:53.958499 (+ 1956us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:55:53.958515 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:55:53.960309 (+ 1794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:55:53.960316 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:55:53.960461 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:55:53.960494 (+ 33us) client_negotiation.cc:769] Sending connection context
0809 22:55:53.960646 (+ 152us) client_negotiation.cc:240] Negotiation successful
0809 22:55:53.960779 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2460,"thread_start_us":2320,"threads_started":1}
I20260809 22:55:53.961939 6275 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b274f20c33db4e75bad54d9e65792cd7" candidate_uuid: "6fb2492799864fb8b944c12e0aa53c0a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9491fd35ae64be7be6fa206b46d187a" is_pre_election: true
I20260809 22:55:53.962452 6275 raft_consensus.cc:2454] T b274f20c33db4e75bad54d9e65792cd7 P e9491fd35ae64be7be6fa206b46d187a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6fb2492799864fb8b944c12e0aa53c0a in term 0.
I20260809 22:55:53.963063 6387 leader_election.cc:302] T b274f20c33db4e75bad54d9e65792cd7 P 6fb2492799864fb8b944c12e0aa53c0a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6fb2492799864fb8b944c12e0aa53c0a, e9491fd35ae64be7be6fa206b46d187a; no voters:
I20260809 22:55:53.963423 6798 raft_consensus.cc:2781] T b274f20c33db4e75bad54d9e65792cd7 P 6fb2492799864fb8b944c12e0aa53c0a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 22:55:53.963568 6798 raft_consensus.cc:482] T b274f20c33db4e75bad54d9e65792cd7 P 6fb2492799864fb8b944c12e0aa53c0a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 22:55:53.963653 6798 raft_consensus.cc:3037] T b274f20c33db4e75bad54d9e65792cd7 P 6fb2492799864fb8b944c12e0aa53c0a [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:55:53.966864 6798 raft_consensus.cc:504] T b274f20c33db4e75bad54d9e65792cd7 P 6fb2492799864fb8b944c12e0aa53c0a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fb2492799864fb8b944c12e0aa53c0a" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 39003 } } peers { permanent_uuid: "e9491fd35ae64be7be6fa206b46d187a" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 34939 } } peers { permanent_uuid: "82971d28843442a7a9c8338a4ad220bb" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 36817 } }
I20260809 22:55:53.967543 6798 leader_election.cc:288] T b274f20c33db4e75bad54d9e65792cd7 P 6fb2492799864fb8b944c12e0aa53c0a [CANDIDATE]: Term 1 election: Requested vote from peers e9491fd35ae64be7be6fa206b46d187a (127.15.113.129:34939), 82971d28843442a7a9c8338a4ad220bb (127.15.113.131:36817)
I20260809 22:55:53.967945 6275 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b274f20c33db4e75bad54d9e65792cd7" candidate_uuid: "6fb2492799864fb8b944c12e0aa53c0a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9491fd35ae64be7be6fa206b46d187a"
I20260809 22:55:53.968119 6275 raft_consensus.cc:3037] T b274f20c33db4e75bad54d9e65792cd7 P e9491fd35ae64be7be6fa206b46d187a [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:55:53.969386 6788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:55:53.951561 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.113.130:55025 (local address 127.15.113.131:36817)
0809 22:55:53.951741 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0809 22:55:53.951748 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:55:53.955624 (+ 3876us) server_negotiation.cc:407] Connection header received
0809 22:55:53.955686 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:55:53.955690 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:55:53.955757 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:55:53.955840 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:55:53.956777 (+ 937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:55:53.957848 (+ 1071us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:55:53.967255 (+ 9407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:55:53.968455 (+ 1200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:55:53.968604 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:55:53.968683 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0809 22:55:53.969027 (+ 344us) server_negotiation.cc:299] Negotiation successful
0809 22:55:53.969158 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260809 22:55:53.969573 6802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:55:53.951856 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.131:36817 (local address 127.15.113.130:55025)
0809 22:55:53.955069 (+ 3213us) negotiation.cc:107] Waiting for socket to connect
0809 22:55:53.955090 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0809 22:55:53.955218 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:55:53.955867 (+ 649us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:55:53.955872 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:55:53.955898 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:55:53.956244 (+ 346us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:55:53.956256 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:55:53.964981 (+ 8725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:55:53.964990 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:55:53.967033 (+ 2043us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:55:53.967048 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:55:53.968675 (+ 1627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:55:53.968682 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:55:53.968856 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:55:53.968911 (+ 55us) client_negotiation.cc:769] Sending connection context
0809 22:55:53.969006 (+ 95us) client_negotiation.cc:240] Negotiation successful
0809 22:55:53.969135 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3076,"thread_start_us":2921,"threads_started":1}
I20260809 22:55:53.970695 6700 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b274f20c33db4e75bad54d9e65792cd7" candidate_uuid: "6fb2492799864fb8b944c12e0aa53c0a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "82971d28843442a7a9c8338a4ad220bb" is_pre_election: true
I20260809 22:55:53.970712 6699 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b274f20c33db4e75bad54d9e65792cd7" candidate_uuid: "6fb2492799864fb8b944c12e0aa53c0a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "82971d28843442a7a9c8338a4ad220bb"
I20260809 22:55:53.971120 6699 raft_consensus.cc:3037] T b274f20c33db4e75bad54d9e65792cd7 P 82971d28843442a7a9c8338a4ad220bb [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:55:53.971263 6275 raft_consensus.cc:2454] T b274f20c33db4e75bad54d9e65792cd7 P e9491fd35ae64be7be6fa206b46d187a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6fb2492799864fb8b944c12e0aa53c0a in term 1.
I20260809 22:55:53.971657 6387 leader_election.cc:302] T b274f20c33db4e75bad54d9e65792cd7 P 6fb2492799864fb8b944c12e0aa53c0a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6fb2492799864fb8b944c12e0aa53c0a, e9491fd35ae64be7be6fa206b46d187a; no voters:
I20260809 22:55:53.971925 6798 raft_consensus.cc:2781] T b274f20c33db4e75bad54d9e65792cd7 P 6fb2492799864fb8b944c12e0aa53c0a [term 1 FOLLOWER]: Leader election won for term 1
I20260809 22:55:53.972077 6798 raft_consensus.cc:686] T b274f20c33db4e75bad54d9e65792cd7 P 6fb2492799864fb8b944c12e0aa53c0a [term 1 LEADER]: Becoming Leader. State: Replica: 6fb2492799864fb8b944c12e0aa53c0a, State: Running, Role: LEADER
I20260809 22:55:53.972506 6798 consensus_queue.cc:237] T b274f20c33db4e75bad54d9e65792cd7 P 6fb2492799864fb8b944c12e0aa53c0a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fb2492799864fb8b944c12e0aa53c0a" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 39003 } } peers { permanent_uuid: "e9491fd35ae64be7be6fa206b46d187a" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 34939 } } peers { permanent_uuid: "82971d28843442a7a9c8338a4ad220bb" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 36817 } }
I20260809 22:55:53.973948 6699 raft_consensus.cc:2454] T b274f20c33db4e75bad54d9e65792cd7 P 82971d28843442a7a9c8338a4ad220bb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6fb2492799864fb8b944c12e0aa53c0a in term 1.
I20260809 22:55:53.977295 6051 catalog_manager.cc:5495] T b274f20c33db4e75bad54d9e65792cd7 P 6fb2492799864fb8b944c12e0aa53c0a reported cstate change: term changed from 0 to 1, leader changed from <none> to 6fb2492799864fb8b944c12e0aa53c0a (127.15.113.130). New cstate: current_term: 1 leader_uuid: "6fb2492799864fb8b944c12e0aa53c0a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fb2492799864fb8b944c12e0aa53c0a" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 39003 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e9491fd35ae64be7be6fa206b46d187a" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 34939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "82971d28843442a7a9c8338a4ad220bb" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 36817 } health_report { overall_health: UNKNOWN } } }
I20260809 22:55:54.007370 6781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:55:54.002490 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49250 (local address 127.15.113.130:39003)
0809 22:55:54.002663 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0809 22:55:54.002668 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:55:54.002750 (+ 82us) server_negotiation.cc:407] Connection header received
0809 22:55:54.002923 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:55:54.002927 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:55:54.003016 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:55:54.003136 (+ 120us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 22:55:54.003425 (+ 289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:55:54.004111 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:55:54.005509 (+ 1398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:55:54.005920 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:55:54.006007 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:55:54.006414 (+ 407us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 22:55:54.006912 (+ 498us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 22:55:54.007016 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0809 22:55:54.007099 (+ 83us) server_negotiation.cc:299] Negotiation successful
0809 22:55:54.007174 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260809 22:55:54.015445 6699 raft_consensus.cc:1261] T b274f20c33db4e75bad54d9e65792cd7 P 82971d28843442a7a9c8338a4ad220bb [term 1 FOLLOWER]: Refusing update from remote peer 6fb2492799864fb8b944c12e0aa53c0a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 22:55:54.015604 6275 raft_consensus.cc:1261] T b274f20c33db4e75bad54d9e65792cd7 P e9491fd35ae64be7be6fa206b46d187a [term 1 FOLLOWER]: Refusing update from remote peer 6fb2492799864fb8b944c12e0aa53c0a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 22:55:54.016350 6795 consensus_queue.cc:1035] T b274f20c33db4e75bad54d9e65792cd7 P 6fb2492799864fb8b944c12e0aa53c0a [LEADER]: Connected to new peer: Peer: permanent_uuid: "82971d28843442a7a9c8338a4ad220bb" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 36817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 22:55:54.016707 6798 consensus_queue.cc:1035] T b274f20c33db4e75bad54d9e65792cd7 P 6fb2492799864fb8b944c12e0aa53c0a [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9491fd35ae64be7be6fa206b46d187a" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 34939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 22:55:54.028919 6805 mvcc.cc:205] Tried to move back new op lower bound from 7316750966837145600 to 7316750966675755008. Current Snapshot: MvccSnapshot[applied={T|T < 7316750966837145600}]
I20260809 22:55:54.039494 6051 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:58084:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 22:55:54.039772 6051 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:58084:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 22:55:54.043141 6051 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 60c579fec9964c88b77f838d9c13df36: Sending DeleteTablet for 3 replicas of tablet b274f20c33db4e75bad54d9e65792cd7
I20260809 22:55:54.044147 6450 tablet_service.cc:1514] Processing DeleteTablet for tablet b274f20c33db4e75bad54d9e65792cd7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:55:54 UTC) from {username='kudu', principal='kudu/127.15.113.190@KRBTEST.COM'} at 127.0.0.1:49240
I20260809 22:55:54.044529 6665 tablet_service.cc:1514] Processing DeleteTablet for tablet b274f20c33db4e75bad54d9e65792cd7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:55:54 UTC) from {username='kudu', principal='kudu/127.15.113.190@KRBTEST.COM'} at 127.0.0.1:50082
I20260809 22:55:54.044787 15814 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskItYPJR/build/asan/bin/kudu pid 6109
I20260809 22:55:54.044938 6247 tablet_service.cc:1514] Processing DeleteTablet for tablet b274f20c33db4e75bad54d9e65792cd7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:55:54 UTC) from {username='kudu', principal='kudu/127.15.113.190@KRBTEST.COM'} at 127.0.0.1:57716
I20260809 22:55:54.047010 6822 tablet_replica.cc:330] T b274f20c33db4e75bad54d9e65792cd7 P e9491fd35ae64be7be6fa206b46d187a: stopping tablet replica
I20260809 22:55:54.047487 6822 raft_consensus.cc:2229] T b274f20c33db4e75bad54d9e65792cd7 P e9491fd35ae64be7be6fa206b46d187a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 22:55:54.048069 6822 raft_consensus.cc:2258] T b274f20c33db4e75bad54d9e65792cd7 P e9491fd35ae64be7be6fa206b46d187a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 22:55:54.051749 6822 ts_tablet_manager.cc:1905] T b274f20c33db4e75bad54d9e65792cd7 P e9491fd35ae64be7be6fa206b46d187a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 22:55:54.056277 6820 tablet_replica.cc:330] T b274f20c33db4e75bad54d9e65792cd7 P 6fb2492799864fb8b944c12e0aa53c0a: stopping tablet replica
I20260809 22:55:54.056824 6820 raft_consensus.cc:2229] T b274f20c33db4e75bad54d9e65792cd7 P 6fb2492799864fb8b944c12e0aa53c0a [term 1 LEADER]: Raft consensus shutting down.
I20260809 22:55:54.057569 6820 raft_consensus.cc:2258] T b274f20c33db4e75bad54d9e65792cd7 P 6fb2492799864fb8b944c12e0aa53c0a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 22:55:54.057854 6821 tablet_replica.cc:330] T b274f20c33db4e75bad54d9e65792cd7 P 82971d28843442a7a9c8338a4ad220bb: stopping tablet replica
I20260809 22:55:54.058291 6821 raft_consensus.cc:2229] T b274f20c33db4e75bad54d9e65792cd7 P 82971d28843442a7a9c8338a4ad220bb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 22:55:54.058799 6821 raft_consensus.cc:2258] T b274f20c33db4e75bad54d9e65792cd7 P 82971d28843442a7a9c8338a4ad220bb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 22:55:54.059763 6820 ts_tablet_manager.cc:1905] T b274f20c33db4e75bad54d9e65792cd7 P 6fb2492799864fb8b944c12e0aa53c0a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 22:55:54.063052 6821 ts_tablet_manager.cc:1905] T b274f20c33db4e75bad54d9e65792cd7 P 82971d28843442a7a9c8338a4ad220bb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 22:55:54.064147 6783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:55:54.046006 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57732 (local address 127.15.113.129:34939)
0809 22:55:54.046174 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0809 22:55:54.046180 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:55:54.046198 (+ 18us) server_negotiation.cc:407] Connection header received
0809 22:55:54.046254 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:55:54.046263 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:55:54.046336 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:55:54.046446 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:55:54.048638 (+ 2192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:55:54.049791 (+ 1153us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:55:54.051916 (+ 2125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:55:54.052564 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:55:54.052694 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:55:54.056626 (+ 3932us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:55:54.056639 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:55:54.056655 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:55:54.056692 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:55:54.060671 (+ 3979us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:55:54.061713 (+ 1042us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:55:54.061723 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:55:54.061726 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:55:54.061890 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:55:54.062336 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:55:54.062342 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:55:54.062345 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:55:54.062848 (+ 503us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:55:54.063186 (+ 338us) server_negotiation.cc:1035] Waiting for connection context
0809 22:55:54.063513 (+ 327us) server_negotiation.cc:299] Negotiation successful
0809 22:55:54.063742 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260809 22:55:54.065071 6341 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:55:54.066813 6820 ts_tablet_manager.cc:1918] T b274f20c33db4e75bad54d9e65792cd7 P 6fb2492799864fb8b944c12e0aa53c0a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 22:55:54.067041 6820 log.cc:1196] T b274f20c33db4e75bad54d9e65792cd7 P 6fb2492799864fb8b944c12e0aa53c0a: Deleting WAL directory at /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315935617919-15814-0/minicluster-data/ts-1/wal/wals/b274f20c33db4e75bad54d9e65792cd7
I20260809 22:55:54.067744 6820 ts_tablet_manager.cc:1939] T b274f20c33db4e75bad54d9e65792cd7 P 6fb2492799864fb8b944c12e0aa53c0a: Deleting consensus metadata
I20260809 22:55:54.069569 6022 catalog_manager.cc:4841] TS 6fb2492799864fb8b944c12e0aa53c0a (127.15.113.130:39003): tablet b274f20c33db4e75bad54d9e65792cd7 (table test-table [id=a7c0b42e69ea4ca281bbe21670b61a6d]) successfully deleted
I20260809 22:55:54.071246 6821 ts_tablet_manager.cc:1918] T b274f20c33db4e75bad54d9e65792cd7 P 82971d28843442a7a9c8338a4ad220bb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 22:55:54.071420 6821 log.cc:1196] T b274f20c33db4e75bad54d9e65792cd7 P 82971d28843442a7a9c8338a4ad220bb: Deleting WAL directory at /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315935617919-15814-0/minicluster-data/ts-2/wal/wals/b274f20c33db4e75bad54d9e65792cd7
I20260809 22:55:54.071848 6821 ts_tablet_manager.cc:1939] T b274f20c33db4e75bad54d9e65792cd7 P 82971d28843442a7a9c8338a4ad220bb: Deleting consensus metadata
I20260809 22:55:54.073242 6028 catalog_manager.cc:4841] TS 82971d28843442a7a9c8338a4ad220bb (127.15.113.131:36817): tablet b274f20c33db4e75bad54d9e65792cd7 (table test-table [id=a7c0b42e69ea4ca281bbe21670b61a6d]) successfully deleted
I20260809 22:55:54.158206 6822 ts_tablet_manager.cc:1918] T b274f20c33db4e75bad54d9e65792cd7 P e9491fd35ae64be7be6fa206b46d187a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 22:55:54.158496 6822 log.cc:1196] T b274f20c33db4e75bad54d9e65792cd7 P e9491fd35ae64be7be6fa206b46d187a: Deleting WAL directory at /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315935617919-15814-0/minicluster-data/ts-0/wal/wals/b274f20c33db4e75bad54d9e65792cd7
I20260809 22:55:54.159199 6822 ts_tablet_manager.cc:1939] T b274f20c33db4e75bad54d9e65792cd7 P e9491fd35ae64be7be6fa206b46d187a: Deleting consensus metadata
I20260809 22:55:54.161428 15814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskItYPJR/build/asan/bin/kudu with pid 6109
I20260809 22:55:54.161514 6027 catalog_manager.cc:4841] TS e9491fd35ae64be7be6fa206b46d187a (127.15.113.129:34939): tablet b274f20c33db4e75bad54d9e65792cd7 (table test-table [id=a7c0b42e69ea4ca281bbe21670b61a6d]) successfully deleted
I20260809 22:55:54.196130 15814 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskItYPJR/build/asan/bin/kudu pid 6359
I20260809 22:55:54.210418 6781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:55:54.196885 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49266 (local address 127.15.113.130:39003)
0809 22:55:54.197087 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0809 22:55:54.197093 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:55:54.197253 (+ 160us) server_negotiation.cc:407] Connection header received
0809 22:55:54.197431 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:55:54.197435 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:55:54.197532 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:55:54.197616 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:55:54.199022 (+ 1406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:55:54.199851 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:55:54.201254 (+ 1403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:55:54.201732 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:55:54.201848 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:55:54.204453 (+ 2605us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:55:54.204465 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:55:54.204481 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:55:54.204515 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:55:54.207586 (+ 3071us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:55:54.208263 (+ 677us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:55:54.208271 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:55:54.208273 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:55:54.208432 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:55:54.208800 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:55:54.208804 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:55:54.208806 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:55:54.209242 (+ 436us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:55:54.209563 (+ 321us) server_negotiation.cc:1035] Waiting for connection context
0809 22:55:54.209853 (+ 290us) server_negotiation.cc:299] Negotiation successful
0809 22:55:54.210072 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260809 22:55:54.211151 6561 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:55:54.319394 15814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskItYPJR/build/asan/bin/kudu with pid 6359
I20260809 22:55:54.346377 15814 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskItYPJR/build/asan/bin/kudu pid 6572
I20260809 22:55:54.359416 6788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:55:54.347078 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50090 (local address 127.15.113.131:36817)
0809 22:55:54.347262 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0809 22:55:54.347267 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:55:54.347285 (+ 18us) server_negotiation.cc:407] Connection header received
0809 22:55:54.347442 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:55:54.347446 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:55:54.347518 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:55:54.347599 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:55:54.348914 (+ 1315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:55:54.349647 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:55:54.350901 (+ 1254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:55:54.351282 (+ 381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:55:54.351384 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:55:54.354271 (+ 2887us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:55:54.354285 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:55:54.354302 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:55:54.354338 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:55:54.356766 (+ 2428us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:55:54.357443 (+ 677us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:55:54.357451 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:55:54.357453 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:55:54.357585 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:55:54.357941 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:55:54.357945 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:55:54.357947 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:55:54.358296 (+ 349us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:55:54.358577 (+ 281us) server_negotiation.cc:1035] Waiting for connection context
0809 22:55:54.358885 (+ 308us) server_negotiation.cc:299] Negotiation successful
0809 22:55:54.359099 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260809 22:55:54.359994 6756 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:55:54.449081 15814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskItYPJR/build/asan/bin/kudu with pid 6572
I20260809 22:55:54.474624 15814 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskItYPJR/build/asan/bin/kudu pid 5996
I20260809 22:55:54.475677 6100 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:55:54.587371 15814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskItYPJR/build/asan/bin/kudu with pid 5996
I20260809 22:55:54.615814 15814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskItYPJR/build/asan/bin/kudu
/tmp/dist-test-taskItYPJR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315935617919-15814-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315935617919-15814-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315935617919-15814-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315935617919-15814-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.113.190:46137
--webserver_interface=127.15.113.190
--webserver_port=45615
--keytab_file=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.190.keytab
--principal=kudu/127.15.113.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.113.148:34295
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.113.190:46137
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315935617919-15814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315935617919-15814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315935617919-15814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:55:54.946794 6831 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:55:54.947101 6831 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:55:54.947260 6831 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:55:54.954469 6831 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 22:55:54.954591 6831 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:55:54.954676 6831 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:55:54.954762 6831 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 22:55:54.954828 6831 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 22:55:54.964540 6831 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.113.148:34295
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315935617919-15814-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315935617919-15814-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.113.190:46137
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.190.keytab
--principal=kudu/127.15.113.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.113.190:46137
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315935617919-15814-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.113.190
--webserver_port=45615
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315935617919-15814-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:55:54.965837 6831 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:55:54.967226 6831 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:55:54.979131 6837 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:55:54.979295 6839 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:55:54.979592 6836 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:55:54.979954 6831 server_base.cc:1034] running on GCE node
I20260809 22:55:54.980721 6831 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:55:54.982151 6831 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:55:54.983361 6831 hybrid_clock.cc:648] HybridClock initialized: now 1786316154983299 us; error 47 us; skew 500 ppm
I20260809 22:55:54.983700 6831 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:55:54 dist-test-slave-ncp9 krb5kdc[5135](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316154, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.113.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:55:54.986709 6831 init.cc:377] Logged in from keytab as kudu/127.15.113.190@KRBTEST.COM (short username kudu)
I20260809 22:55:54.988119 6831 webserver.cc:460] Webserver started at http://127.15.113.190:45615/ using document root <none> and password file <none>
I20260809 22:55:54.988771 6831 fs_manager.cc:359] Metadata directory not provided
I20260809 22:55:54.988917 6831 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:55:54.994190 6831 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.003s
I20260809 22:55:54.996855 6849 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:55:54.998131 6831 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 22:55:54.998289 6831 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315935617919-15814-0/minicluster-data/master-0/data,/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315935617919-15814-0/minicluster-data/master-0/wal
uuid: "60c579fec9964c88b77f838d9c13df36"
format_stamp: "Formatted at 2026-08-09 22:55:49 on dist-test-slave-ncp9"
server_key: "b1fa6de1da0146b96208386a306420e6"
server_key_iv: "1740c3981367fca234f4799be5a4c375"
server_key_version: "encryptionkey@0"
I20260809 22:55:54.999233 6831 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315935617919-15814-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315935617919-15814-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315935617919-15814-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:55:55.015645 6831 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:55:55.020061 6831 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:55:55.020284 6831 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:55:55.036489 6831 rpc_server.cc:307] RPC server started. Bound to: 127.15.113.190:46137
I20260809 22:55:55.036509 6920 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.113.190:46137 every 8 connection(s)
I20260809 22:55:55.037972 6831 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315935617919-15814-0/minicluster-data/master-0/data/info.pb
I20260809 22:55:55.041447 15814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskItYPJR/build/asan/bin/kudu as pid 6831
I20260809 22:55:55.042429 15814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskItYPJR/build/asan/bin/kudu
/tmp/dist-test-taskItYPJR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315935617919-15814-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315935617919-15814-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315935617919-15814-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315935617919-15814-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.113.129:34939
--local_ip_for_outbound_sockets=127.15.113.129
--tserver_master_addrs=127.15.113.190:46137
--webserver_port=45525
--webserver_interface=127.15.113.129
--keytab_file=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.129.keytab
--principal=kudu/127.15.113.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.113.148:34295
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315935617919-15814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315935617919-15814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315935617919-15814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 22:55:55.045872 6921 sys_catalog.cc:263] Verifying existing consensus state
I20260809 22:55:55.050024 6921 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 60c579fec9964c88b77f838d9c13df36: Bootstrap starting.
I20260809 22:55:55.070447 6921 log.cc:824] T 00000000000000000000000000000000 P 60c579fec9964c88b77f838d9c13df36: Log is configured to *not* fsync() on all Append() calls
I20260809 22:55:55.086233 6921 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 60c579fec9964c88b77f838d9c13df36: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 22:55:55.086737 6921 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 60c579fec9964c88b77f838d9c13df36: Bootstrap complete.
I20260809 22:55:55.095849 6921 raft_consensus.cc:348] T 00000000000000000000000000000000 P 60c579fec9964c88b77f838d9c13df36 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60c579fec9964c88b77f838d9c13df36" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 46137 } }
I20260809 22:55:55.096712 6921 raft_consensus.cc:729] T 00000000000000000000000000000000 P 60c579fec9964c88b77f838d9c13df36 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60c579fec9964c88b77f838d9c13df36, State: Initialized, Role: FOLLOWER
I20260809 22:55:55.097370 6921 consensus_queue.cc:260] T 00000000000000000000000000000000 P 60c579fec9964c88b77f838d9c13df36 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60c579fec9964c88b77f838d9c13df36" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 46137 } }
I20260809 22:55:55.097582 6921 raft_consensus.cc:388] T 00000000000000000000000000000000 P 60c579fec9964c88b77f838d9c13df36 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 22:55:55.097710 6921 raft_consensus.cc:482] T 00000000000000000000000000000000 P 60c579fec9964c88b77f838d9c13df36 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 22:55:55.097874 6921 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 60c579fec9964c88b77f838d9c13df36 [term 2 FOLLOWER]: Advancing to term 3
I20260809 22:55:55.101665 6921 raft_consensus.cc:504] T 00000000000000000000000000000000 P 60c579fec9964c88b77f838d9c13df36 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60c579fec9964c88b77f838d9c13df36" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 46137 } }
I20260809 22:55:55.102177 6921 leader_election.cc:302] T 00000000000000000000000000000000 P 60c579fec9964c88b77f838d9c13df36 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 60c579fec9964c88b77f838d9c13df36; no voters:
I20260809 22:55:55.102849 6921 leader_election.cc:288] T 00000000000000000000000000000000 P 60c579fec9964c88b77f838d9c13df36 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260809 22:55:55.102993 6926 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 60c579fec9964c88b77f838d9c13df36 [term 3 FOLLOWER]: Leader election won for term 3
I20260809 22:55:55.104022 6926 raft_consensus.cc:686] T 00000000000000000000000000000000 P 60c579fec9964c88b77f838d9c13df36 [term 3 LEADER]: Becoming Leader. State: Replica: 60c579fec9964c88b77f838d9c13df36, State: Running, Role: LEADER
I20260809 22:55:55.104638 6926 consensus_queue.cc:237] T 00000000000000000000000000000000 P 60c579fec9964c88b77f838d9c13df36 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60c579fec9964c88b77f838d9c13df36" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 46137 } }
I20260809 22:55:55.104972 6921 sys_catalog.cc:564] T 00000000000000000000000000000000 P 60c579fec9964c88b77f838d9c13df36 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 22:55:55.107151 6929 sys_catalog.cc:455] T 00000000000000000000000000000000 P 60c579fec9964c88b77f838d9c13df36 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 60c579fec9964c88b77f838d9c13df36. Latest consensus state: current_term: 3 leader_uuid: "60c579fec9964c88b77f838d9c13df36" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60c579fec9964c88b77f838d9c13df36" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 46137 } } }
I20260809 22:55:55.107515 6929 sys_catalog.cc:458] T 00000000000000000000000000000000 P 60c579fec9964c88b77f838d9c13df36 [sys.catalog]: This master's current role is: LEADER
I20260809 22:55:55.108016 6927 sys_catalog.cc:455] T 00000000000000000000000000000000 P 60c579fec9964c88b77f838d9c13df36 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "60c579fec9964c88b77f838d9c13df36" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60c579fec9964c88b77f838d9c13df36" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 46137 } } }
I20260809 22:55:55.108294 6927 sys_catalog.cc:458] T 00000000000000000000000000000000 P 60c579fec9964c88b77f838d9c13df36 [sys.catalog]: This master's current role is: LEADER
I20260809 22:55:55.113106 6935 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 22:55:55.121735 6935 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 22:55:55.123319 6935 catalog_manager.cc:1259] Loaded cluster ID: 85ec813c10da49099b9340d15162601d
I20260809 22:55:55.123468 6935 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260809 22:55:55.125681 6935 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7fc63de029ad google::LogMessage::Fail() at ??:0
@ 0x7fc63de0588a google::LogMessage::SendToLog() at ??:0
@ 0x7fc63de024de google::LogMessage::Flush() at ??:0
@ 0x7fc63de03a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7fc653835512 kudu::master::CatalogManager::PrepareForLeadershipTask()::$_6::operator()() at ??:0
@ 0x7fc653832976 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7fc6538ba610 kudu::master::CatalogManager::ElectedAsLeaderCb()::$_2::operator()() at ??:0
@ 0x7fc6538ba439 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7fc640a4e345 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7fc640a5191c kudu::ThreadPool::CreateThread()::$_2::operator()() at ??:0
@ 0x7fc640a51799 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7fc640a2ceaf kudu::Thread::SuperviseThread() at ??:0
@ 0x7fc643d066db start_thread at ??:0
@ 0x7fc63cc9c61f clone at ??:0
W20260809 22:55:55.461306 6924 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:55:55.461556 6924 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:55:55.461658 6924 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:55:55.468554 6924 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:55:55.468654 6924 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:55:55.468775 6924 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.113.129
I20260809 22:55:55.477986 6924 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.113.148:34295
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315935617919-15814-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315935617919-15814-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.129.keytab
--principal=kudu/127.15.113.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.113.129:34939
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315935617919-15814-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.113.129
--webserver_port=45525
--tserver_master_addrs=127.15.113.190:46137
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.113.129
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315935617919-15814-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:55:55.479106 6924 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:55:55.480370 6924 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:55:55.493053 6977 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:55:55.493021 6978 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:55:55.493229 6981 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:55:55.494153 6924 server_base.cc:1034] running on GCE node
I20260809 22:55:55.494763 6924 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:55:55.495649 6924 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:55:55.496845 6924 hybrid_clock.cc:648] HybridClock initialized: now 1786316155496796 us; error 38 us; skew 500 ppm
I20260809 22:55:55.497155 6924 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:55:55 dist-test-slave-ncp9 krb5kdc[5135](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316155, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.113.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:55:55.500089 6924 init.cc:377] Logged in from keytab as kudu/127.15.113.129@KRBTEST.COM (short username kudu)
I20260809 22:55:55.501543 6924 webserver.cc:460] Webserver started at http://127.15.113.129:45525/ using document root <none> and password file <none>
I20260809 22:55:55.502238 6924 fs_manager.cc:359] Metadata directory not provided
I20260809 22:55:55.502362 6924 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:55:55.507537 6924 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260809 22:55:55.510170 6992 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:55:55.511389 6924 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 22:55:55.511536 6924 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315935617919-15814-0/minicluster-data/ts-0/data,/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315935617919-15814-0/minicluster-data/ts-0/wal
uuid: "e9491fd35ae64be7be6fa206b46d187a"
format_stamp: "Formatted at 2026-08-09 22:55:49 on dist-test-slave-ncp9"
server_key: "688d8c1cc62f3dd447fd01428a4507b3"
server_key_iv: "1f230b0c43cfd79a3fc22fe895743930"
server_key_version: "encryptionkey@0"
I20260809 22:55:55.512478 6924 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315935617919-15814-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315935617919-15814-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315935617919-15814-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:55:55.543715 6924 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:55:55.547572 6924 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:55:55.547829 6924 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:55:55.548807 6924 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:55:55.550122 6924 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:55:55.550226 6924 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:55:55.550335 6924 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:55:55.550411 6924 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:55:55.581357 6924 rpc_server.cc:307] RPC server started. Bound to: 127.15.113.129:34939
I20260809 22:55:55.581430 7159 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.113.129:34939 every 8 connection(s)
I20260809 22:55:55.582768 6924 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315935617919-15814-0/minicluster-data/ts-0/data/info.pb
I20260809 22:55:55.587483 7162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:55:55.585637 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:46137 (local address 127.15.113.129:49567)
0809 22:55:55.586352 (+ 715us) negotiation.cc:107] Waiting for socket to connect
0809 22:55:55.586574 (+ 222us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.113.190:46137: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":459,"thread_start_us":268,"threads_started":1}
W20260809 22:55:55.588074 7160 heartbeater.cc:644] Failed to heartbeat to 127.15.113.190:46137 (0 consecutive failures): Network error: Failed to ping master at 127.15.113.190:46137: Client connection negotiation failed: client connection to 127.15.113.190:46137: connect: Connection refused (error 111)
I20260809 22:55:55.588809 7162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:55:55.588516 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:46137 (local address 127.15.113.129:52717)
0809 22:55:55.588643 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0809 22:55:55.588707 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.113.190:46137: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260809 22:55:55.589526 7162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:55:55.589260 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:46137 (local address 127.15.113.129:35975)
0809 22:55:55.589359 (+ 99us) negotiation.cc:107] Waiting for socket to connect
0809 22:55:55.589427 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.113.190:46137: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260809 22:55:55.589749 7160 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260809 22:55:55.592273 15814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskItYPJR/build/asan/bin/kudu as pid 6924
I20260809 22:55:55.592857 15814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskItYPJR/build/asan/bin/kudu
/tmp/dist-test-taskItYPJR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315935617919-15814-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315935617919-15814-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315935617919-15814-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315935617919-15814-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.113.130:39003
--local_ip_for_outbound_sockets=127.15.113.130
--tserver_master_addrs=127.15.113.190:46137
--webserver_port=40721
--webserver_interface=127.15.113.130
--keytab_file=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.130.keytab
--principal=kudu/127.15.113.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.113.148:34295
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315935617919-15814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315935617919-15814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315935617919-15814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:55:55.920825 7163 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:55:55.921118 7163 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:55:55.921259 7163 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:55:55.928308 7163 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:55:55.928421 7163 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:55:55.928614 7163 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.113.130
I20260809 22:55:55.938081 7163 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.113.148:34295
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315935617919-15814-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315935617919-15814-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.130.keytab
--principal=kudu/127.15.113.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.113.130:39003
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315935617919-15814-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.113.130
--webserver_port=40721
--tserver_master_addrs=127.15.113.190:46137
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.113.130
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315935617919-15814-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:55:55.939329 7163 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:55:55.940608 7163 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:55:55.953173 7173 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:55:55.953249 7168 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:55:55.953223 7169 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:55:55.954383 7163 server_base.cc:1034] running on GCE node
I20260809 22:55:55.954988 7163 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:55:55.955847 7163 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:55:55.957079 7163 hybrid_clock.cc:648] HybridClock initialized: now 1786316155956992 us; error 35 us; skew 500 ppm
I20260809 22:55:55.957432 7163 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:55:55 dist-test-slave-ncp9 krb5kdc[5135](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316155, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.113.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:55:55.960366 7163 init.cc:377] Logged in from keytab as kudu/127.15.113.130@KRBTEST.COM (short username kudu)
I20260809 22:55:55.961702 7163 webserver.cc:460] Webserver started at http://127.15.113.130:40721/ using document root <none> and password file <none>
I20260809 22:55:55.962363 7163 fs_manager.cc:359] Metadata directory not provided
I20260809 22:55:55.962486 7163 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:55:55.967556 7163 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260809 22:55:55.970131 7187 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:55:55.971356 7163 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 22:55:55.971505 7163 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315935617919-15814-0/minicluster-data/ts-1/data,/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315935617919-15814-0/minicluster-data/ts-1/wal
uuid: "6fb2492799864fb8b944c12e0aa53c0a"
format_stamp: "Formatted at 2026-08-09 22:55:50 on dist-test-slave-ncp9"
server_key: "3bd00e7d274a6999ef76a1d8237a5ab7"
server_key_iv: "29b30cacc051bf72a62afb639737ac6c"
server_key_version: "encryptionkey@0"
I20260809 22:55:55.972458 7163 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315935617919-15814-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315935617919-15814-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315935617919-15814-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:55:55.996341 7163 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:55:56.000329 7163 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:55:56.000583 7163 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:55:56.001578 7163 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:55:56.002910 7163 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:55:56.003006 7163 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:55:56.003121 7163 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:55:56.003191 7163 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:55:56.033947 7163 rpc_server.cc:307] RPC server started. Bound to: 127.15.113.130:39003
I20260809 22:55:56.033993 7342 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.113.130:39003 every 8 connection(s)
I20260809 22:55:56.035498 7163 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315935617919-15814-0/minicluster-data/ts-1/data/info.pb
I20260809 22:55:56.039345 15814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskItYPJR/build/asan/bin/kudu as pid 7163
I20260809 22:55:56.040072 15814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskItYPJR/build/asan/bin/kudu
/tmp/dist-test-taskItYPJR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315935617919-15814-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315935617919-15814-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315935617919-15814-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315935617919-15814-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.113.131:36817
--local_ip_for_outbound_sockets=127.15.113.131
--tserver_master_addrs=127.15.113.190:46137
--webserver_port=41869
--webserver_interface=127.15.113.131
--keytab_file=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.131.keytab
--principal=kudu/127.15.113.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.113.148:34295
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315935617919-15814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315935617919-15814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315935617919-15814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 22:55:56.040441 7347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:55:56.038523 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:46137 (local address 127.15.113.130:34727)
0809 22:55:56.039272 (+ 749us) negotiation.cc:107] Waiting for socket to connect
0809 22:55:56.039514 (+ 242us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.113.190:46137: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":493,"thread_start_us":308,"threads_started":1}
W20260809 22:55:56.041162 7343 heartbeater.cc:644] Failed to heartbeat to 127.15.113.190:46137 (0 consecutive failures): Network error: Failed to ping master at 127.15.113.190:46137: Client connection negotiation failed: client connection to 127.15.113.190:46137: connect: Connection refused (error 111)
I20260809 22:55:56.041977 7347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:55:56.041656 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:46137 (local address 127.15.113.130:36833)
0809 22:55:56.041797 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0809 22:55:56.041863 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.113.190:46137: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260809 22:55:56.042757 7347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:55:56.042456 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:46137 (local address 127.15.113.130:49929)
0809 22:55:56.042592 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0809 22:55:56.042660 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.113.190:46137: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260809 22:55:56.042981 7343 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260809 22:55:56.381089 7348 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:55:56.381366 7348 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:55:56.381488 7348 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:55:56.388748 7348 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:55:56.388852 7348 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:55:56.389012 7348 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.113.131
I20260809 22:55:56.398480 7348 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.113.148:34295
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315935617919-15814-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315935617919-15814-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.131.keytab
--principal=kudu/127.15.113.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.113.131:36817
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315935617919-15814-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.113.131
--webserver_port=41869
--tserver_master_addrs=127.15.113.190:46137
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.113.131
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315935617919-15814-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:55:56.399631 7348 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:55:56.401007 7348 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:55:56.414582 7355 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:55:56.414634 7358 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:55:56.414568 7356 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:55:56.415704 7348 server_base.cc:1034] running on GCE node
I20260809 22:55:56.416368 7348 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:55:56.417340 7348 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:55:56.418535 7348 hybrid_clock.cc:648] HybridClock initialized: now 1786316156418486 us; error 44 us; skew 500 ppm
I20260809 22:55:56.418833 7348 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:55:56 dist-test-slave-ncp9 krb5kdc[5135](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316156, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.113.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:55:56.421876 7348 init.cc:377] Logged in from keytab as kudu/127.15.113.131@KRBTEST.COM (short username kudu)
I20260809 22:55:56.423213 7348 webserver.cc:460] Webserver started at http://127.15.113.131:41869/ using document root <none> and password file <none>
I20260809 22:55:56.423813 7348 fs_manager.cc:359] Metadata directory not provided
I20260809 22:55:56.423914 7348 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:55:56.429067 7348 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.001s
I20260809 22:55:56.431746 7367 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:55:56.433064 7348 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 22:55:56.433202 7348 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315935617919-15814-0/minicluster-data/ts-2/data,/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315935617919-15814-0/minicluster-data/ts-2/wal
uuid: "82971d28843442a7a9c8338a4ad220bb"
format_stamp: "Formatted at 2026-08-09 22:55:50 on dist-test-slave-ncp9"
server_key: "159f3a218e0536600f31a263f5e25b04"
server_key_iv: "0359d24e8ea6f74a0f00667a35524811"
server_key_version: "encryptionkey@0"
I20260809 22:55:56.434090 7348 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315935617919-15814-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315935617919-15814-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315935617919-15814-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:55:56.452620 7348 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:55:56.456571 7348 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:55:56.456782 7348 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:55:56.457724 7348 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:55:56.458967 7348 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:55:56.459052 7348 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:55:56.459147 7348 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:55:56.459203 7348 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:55:56.489753 7348 rpc_server.cc:307] RPC server started. Bound to: 127.15.113.131:36817
I20260809 22:55:56.489817 7530 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.113.131:36817 every 8 connection(s)
I20260809 22:55:56.491151 7348 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315935617919-15814-0/minicluster-data/ts-2/data/info.pb
I20260809 22:55:56.495682 7534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:55:56.493956 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:46137 (local address 127.15.113.131:33567)
0809 22:55:56.494649 (+ 693us) negotiation.cc:107] Waiting for socket to connect
0809 22:55:56.494864 (+ 215us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.113.190:46137: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":448,"thread_start_us":279,"threads_started":1}
W20260809 22:55:56.496259 7531 heartbeater.cc:644] Failed to heartbeat to 127.15.113.190:46137 (0 consecutive failures): Network error: Failed to ping master at 127.15.113.190:46137: Client connection negotiation failed: client connection to 127.15.113.190:46137: connect: Connection refused (error 111)
I20260809 22:55:56.496960 7534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:55:56.496675 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:46137 (local address 127.15.113.131:60227)
0809 22:55:56.496789 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0809 22:55:56.496850 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.113.190:46137: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260809 22:55:56.497673 7534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:55:56.497410 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:46137 (local address 127.15.113.131:41241)
0809 22:55:56.497513 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0809 22:55:56.497578 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.113.190:46137: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260809 22:55:56.497877 7531 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260809 22:55:56.498425 15814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskItYPJR/build/asan/bin/kudu as pid 7348
I20260809 22:55:56.590960 7536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:55:56.590200 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:46137 (local address 127.15.113.129:55961)
0809 22:55:56.590618 (+ 418us) negotiation.cc:107] Waiting for socket to connect
0809 22:55:56.590742 (+ 124us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.113.190:46137: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":316,"thread_start_us":193,"threads_started":1}
I20260809 22:55:57.044167 7537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:55:57.043415 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:46137 (local address 127.15.113.130:56633)
0809 22:55:57.043838 (+ 423us) negotiation.cc:107] Waiting for socket to connect
0809 22:55:57.043947 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.113.190:46137: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":325,"thread_start_us":201,"threads_started":1}
I20260809 22:55:57.499058 7538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:55:57.498315 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:46137 (local address 127.15.113.131:46901)
0809 22:55:57.498723 (+ 408us) negotiation.cc:107] Waiting for socket to connect
0809 22:55:57.498838 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.113.190:46137: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":312,"thread_start_us":197,"threads_started":1}
I20260809 22:55:57.592392 7539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:55:57.591734 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:46137 (local address 127.15.113.129:52579)
0809 22:55:57.592152 (+ 418us) negotiation.cc:107] Waiting for socket to connect
0809 22:55:57.592230 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.113.190:46137: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":320,"thread_start_us":202,"threads_started":1}
I20260809 22:55:58.045679 7541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:55:58.045004 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:46137 (local address 127.15.113.130:44429)
0809 22:55:58.045435 (+ 431us) negotiation.cc:107] Waiting for socket to connect
0809 22:55:58.045511 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.113.190:46137: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":338,"thread_start_us":210,"threads_started":1}
I20260809 22:55:58.500495 7542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:55:58.499826 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:46137 (local address 127.15.113.131:50933)
0809 22:55:58.500253 (+ 427us) negotiation.cc:107] Waiting for socket to connect
0809 22:55:58.500332 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.113.190:46137: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":335,"thread_start_us":207,"threads_started":1}
I20260809 22:55:58.593822 7543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:55:58.593135 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:46137 (local address 127.15.113.129:57655)
0809 22:55:58.593571 (+ 436us) negotiation.cc:107] Waiting for socket to connect
0809 22:55:58.593657 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.113.190:46137: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":346,"thread_start_us":228,"threads_started":1}
I20260809 22:55:59.047086 7544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:55:59.046381 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:46137 (local address 127.15.113.130:49551)
0809 22:55:59.046831 (+ 450us) negotiation.cc:107] Waiting for socket to connect
0809 22:55:59.046906 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.113.190:46137: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":355,"thread_start_us":248,"threads_started":1}
I20260809 22:55:59.501943 7545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:55:59.501285 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:46137 (local address 127.15.113.131:49361)
0809 22:55:59.501704 (+ 419us) negotiation.cc:107] Waiting for socket to connect
0809 22:55:59.501782 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.113.190:46137: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":319,"thread_start_us":221,"threads_started":1}
I20260809 22:55:59.595250 7546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:55:59.594571 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:46137 (local address 127.15.113.129:54645)
0809 22:55:59.594990 (+ 419us) negotiation.cc:107] Waiting for socket to connect
0809 22:55:59.595070 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.113.190:46137: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":324,"thread_start_us":214,"threads_started":1}
I20260809 22:56:00.048573 7547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:56:00.047850 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:46137 (local address 127.15.113.130:35119)
0809 22:56:00.048304 (+ 454us) negotiation.cc:107] Waiting for socket to connect
0809 22:56:00.048387 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.113.190:46137: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":352,"thread_start_us":218,"threads_started":1}
I20260809 22:56:00.503494 7548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:56:00.502732 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:46137 (local address 127.15.113.131:45761)
0809 22:56:00.503209 (+ 477us) negotiation.cc:107] Waiting for socket to connect
0809 22:56:00.503293 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.113.190:46137: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":366,"thread_start_us":241,"threads_started":1}
I20260809 22:56:00.596753 7551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:56:00.596067 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:46137 (local address 127.15.113.129:47535)
0809 22:56:00.596502 (+ 435us) negotiation.cc:107] Waiting for socket to connect
0809 22:56:00.596583 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.113.190:46137: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":340,"thread_start_us":220,"threads_started":1}
I20260809 22:56:01.050091 7552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:56:01.049393 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:46137 (local address 127.15.113.130:44563)
0809 22:56:01.049849 (+ 456us) negotiation.cc:107] Waiting for socket to connect
0809 22:56:01.049927 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.113.190:46137: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":361,"thread_start_us":221,"threads_started":1}
I20260809 22:56:01.505003 7553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:56:01.504307 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:46137 (local address 127.15.113.131:46359)
0809 22:56:01.504727 (+ 420us) negotiation.cc:107] Waiting for socket to connect
0809 22:56:01.504804 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.113.190:46137: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":330,"thread_start_us":208,"threads_started":1}
I20260809 22:56:01.598140 7554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:56:01.597489 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:46137 (local address 127.15.113.129:33357)
0809 22:56:01.597900 (+ 411us) negotiation.cc:107] Waiting for socket to connect
0809 22:56:01.597987 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.113.190:46137: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":318,"thread_start_us":206,"threads_started":1}
I20260809 22:56:02.051510 7557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:56:02.050859 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:46137 (local address 127.15.113.130:56945)
0809 22:56:02.051270 (+ 411us) negotiation.cc:107] Waiting for socket to connect
0809 22:56:02.051356 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.113.190:46137: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":320,"thread_start_us":210,"threads_started":1}
I20260809 22:56:02.506510 7558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:56:02.505831 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:46137 (local address 127.15.113.131:47561)
0809 22:56:02.506263 (+ 432us) negotiation.cc:107] Waiting for socket to connect
0809 22:56:02.506353 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.113.190:46137: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":337,"thread_start_us":215,"threads_started":1}
I20260809 22:56:02.599690 7560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:56:02.598938 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:46137 (local address 127.15.113.129:44331)
0809 22:56:02.599402 (+ 464us) negotiation.cc:107] Waiting for socket to connect
0809 22:56:02.599485 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.113.190:46137: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":363,"thread_start_us":206,"threads_started":1}
I20260809 22:56:03.053092 7562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:56:03.052301 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:46137 (local address 127.15.113.130:54681)
0809 22:56:03.052769 (+ 468us) negotiation.cc:107] Waiting for socket to connect
0809 22:56:03.052847 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.113.190:46137: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":365,"thread_start_us":202,"threads_started":1}
I20260809 22:56:03.508080 7565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:56:03.507288 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:46137 (local address 127.15.113.131:41305)
0809 22:56:03.507795 (+ 507us) negotiation.cc:107] Waiting for socket to connect
0809 22:56:03.507875 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.113.190:46137: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":404,"thread_start_us":230,"threads_started":1}
I20260809 22:56:03.601254 7567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:56:03.600541 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:46137 (local address 127.15.113.129:48637)
0809 22:56:03.601008 (+ 467us) negotiation.cc:107] Waiting for socket to connect
0809 22:56:03.601091 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.113.190:46137: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":371,"thread_start_us":235,"threads_started":1}
I20260809 22:56:04.054584 7571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:56:04.053905 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:46137 (local address 127.15.113.130:32791)
0809 22:56:04.054349 (+ 444us) negotiation.cc:107] Waiting for socket to connect
0809 22:56:04.054428 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.113.190:46137: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":349,"thread_start_us":226,"threads_started":1}
I20260809 22:56:04.509567 7573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:56:04.508867 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:46137 (local address 127.15.113.131:49377)
0809 22:56:04.509321 (+ 454us) negotiation.cc:107] Waiting for socket to connect
0809 22:56:04.509402 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.113.190:46137: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":360,"thread_start_us":212,"threads_started":1}
I20260809 22:56:04.602926 7574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:56:04.602049 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:46137 (local address 127.15.113.129:41275)
0809 22:56:04.602584 (+ 535us) negotiation.cc:107] Waiting for socket to connect
0809 22:56:04.602704 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.113.190:46137: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":432,"thread_start_us":273,"threads_started":1}
I20260809 22:56:05.056193 7578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:56:05.055468 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:46137 (local address 127.15.113.130:56727)
0809 22:56:05.055943 (+ 475us) negotiation.cc:107] Waiting for socket to connect
0809 22:56:05.056022 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.113.190:46137: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":374,"thread_start_us":231,"threads_started":1}
I20260809 22:56:05.511008 7580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:56:05.510319 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:46137 (local address 127.15.113.131:46233)
0809 22:56:05.510763 (+ 444us) negotiation.cc:107] Waiting for socket to connect
0809 22:56:05.510840 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.113.190:46137: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":345,"thread_start_us":219,"threads_started":1}
I20260809 22:56:05.604472 7581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:56:05.603759 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:46137 (local address 127.15.113.129:43701)
0809 22:56:05.604228 (+ 469us) negotiation.cc:107] Waiting for socket to connect
0809 22:56:05.604310 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.113.190:46137: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":364,"thread_start_us":250,"threads_started":1}
I20260809 22:56:06.057771 7583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:56:06.057022 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:46137 (local address 127.15.113.130:53949)
0809 22:56:06.057533 (+ 511us) negotiation.cc:107] Waiting for socket to connect
0809 22:56:06.057613 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.113.190:46137: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":411,"thread_start_us":305,"threads_started":1}
I20260809 22:56:06.512485 7584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:56:06.511844 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:46137 (local address 127.15.113.131:53423)
0809 22:56:06.512253 (+ 409us) negotiation.cc:107] Waiting for socket to connect
0809 22:56:06.512332 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.113.190:46137: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":319,"thread_start_us":219,"threads_started":1}
I20260809 22:56:06.605975 7585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:56:06.605280 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:46137 (local address 127.15.113.129:41959)
0809 22:56:06.605709 (+ 429us) negotiation.cc:107] Waiting for socket to connect
0809 22:56:06.605798 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.113.190:46137: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":327,"thread_start_us":211,"threads_started":1}
I20260809 22:56:07.059386 7586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:56:07.058590 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:46137 (local address 127.15.113.130:37543)
0809 22:56:07.059100 (+ 510us) negotiation.cc:107] Waiting for socket to connect
0809 22:56:07.059211 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.113.190:46137: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":402,"thread_start_us":266,"threads_started":1}
I20260809 22:56:07.513970 7588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:56:07.513273 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:46137 (local address 127.15.113.131:57927)
0809 22:56:07.513705 (+ 432us) negotiation.cc:107] Waiting for socket to connect
0809 22:56:07.513796 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.113.190:46137: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":325,"thread_start_us":203,"threads_started":1}
I20260809 22:56:07.607467 7590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:56:07.606812 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:46137 (local address 127.15.113.129:46701)
0809 22:56:07.607232 (+ 420us) negotiation.cc:107] Waiting for socket to connect
0809 22:56:07.607313 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.113.190:46137: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":306,"thread_start_us":206,"threads_started":1}
I20260809 22:56:08.061012 7591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:56:08.060232 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:46137 (local address 127.15.113.130:55989)
0809 22:56:08.060737 (+ 505us) negotiation.cc:107] Waiting for socket to connect
0809 22:56:08.060820 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.113.190:46137: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":365,"thread_start_us":258,"threads_started":1}
I20260809 22:56:08.515581 7593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:56:08.514872 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:46137 (local address 127.15.113.131:49375)
0809 22:56:08.515316 (+ 444us) negotiation.cc:107] Waiting for socket to connect
0809 22:56:08.515405 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.113.190:46137: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":339,"thread_start_us":226,"threads_started":1}
I20260809 22:56:08.609122 7594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:56:08.608283 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:46137 (local address 127.15.113.129:32961)
0809 22:56:08.608783 (+ 500us) negotiation.cc:107] Waiting for socket to connect
0809 22:56:08.608875 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.113.190:46137: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":396,"thread_start_us":253,"threads_started":1}
I20260809 22:56:09.062562 7598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:56:09.061810 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:46137 (local address 127.15.113.130:42743)
0809 22:56:09.062292 (+ 482us) negotiation.cc:107] Waiting for socket to connect
0809 22:56:09.062381 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.113.190:46137: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":389,"thread_start_us":233,"threads_started":1}
I20260809 22:56:09.517298 7599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:56:09.516476 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:46137 (local address 127.15.113.131:37155)
0809 22:56:09.516996 (+ 520us) negotiation.cc:107] Waiting for socket to connect
0809 22:56:09.517090 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.113.190:46137: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":401,"thread_start_us":224,"threads_started":1}
I20260809 22:56:09.610697 7600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:56:09.609940 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:46137 (local address 127.15.113.129:54777)
0809 22:56:09.610414 (+ 474us) negotiation.cc:107] Waiting for socket to connect
0809 22:56:09.610500 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.113.190:46137: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":371,"thread_start_us":256,"threads_started":1}
I20260809 22:56:10.064239 7603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:56:10.063423 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:46137 (local address 127.15.113.130:33175)
0809 22:56:10.063929 (+ 506us) negotiation.cc:107] Waiting for socket to connect
0809 22:56:10.064010 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.113.190:46137: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":392,"thread_start_us":262,"threads_started":1}
I20260809 22:56:10.518800 7604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:56:10.518112 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:46137 (local address 127.15.113.131:35259)
0809 22:56:10.518541 (+ 429us) negotiation.cc:107] Waiting for socket to connect
0809 22:56:10.518622 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.113.190:46137: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":332,"thread_start_us":221,"threads_started":1}
I20260809 22:56:10.612394 7607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:56:10.611673 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:46137 (local address 127.15.113.129:48795)
0809 22:56:10.612102 (+ 429us) negotiation.cc:107] Waiting for socket to connect
0809 22:56:10.612209 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.113.190:46137: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":326,"thread_start_us":221,"threads_started":1}
I20260809 22:56:11.066051 7608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:56:11.065282 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:46137 (local address 127.15.113.130:34951)
0809 22:56:11.065757 (+ 475us) negotiation.cc:107] Waiting for socket to connect
0809 22:56:11.065871 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.113.190:46137: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":360,"thread_start_us":247,"threads_started":1}
I20260809 22:56:11.507814 15814 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskItYPJR/build/asan/bin/kudu pid 6924
I20260809 22:56:11.520586 7610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:56:11.519836 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:46137 (local address 127.15.113.131:44099)
0809 22:56:11.520302 (+ 466us) negotiation.cc:107] Waiting for socket to connect
0809 22:56:11.520402 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.113.190:46137: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":354,"thread_start_us":235,"threads_started":1}
I20260809 22:56:11.530277 7609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:56:11.508989 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33744 (local address 127.15.113.129:34939)
0809 22:56:11.509715 (+ 726us) server_negotiation.cc:206] Beginning negotiation
0809 22:56:11.509745 (+ 30us) server_negotiation.cc:399] Waiting for connection header
0809 22:56:11.509797 (+ 52us) server_negotiation.cc:407] Connection header received
0809 22:56:11.511649 (+ 1852us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:56:11.511660 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:56:11.512123 (+ 463us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:56:11.512432 (+ 309us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:56:11.513894 (+ 1462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:56:11.515241 (+ 1347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:56:11.516509 (+ 1268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:56:11.517390 (+ 881us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:56:11.517575 (+ 185us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:56:11.524473 (+ 6898us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:56:11.524482 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:56:11.524500 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:56:11.524540 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:56:11.527436 (+ 2896us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:56:11.528006 (+ 570us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:56:11.528011 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:56:11.528018 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:56:11.528206 (+ 188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:56:11.528486 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:56:11.528489 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:56:11.528491 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:56:11.529054 (+ 563us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:56:11.529303 (+ 249us) server_negotiation.cc:1035] Waiting for connection context
0809 22:56:11.529603 (+ 300us) server_negotiation.cc:299] Negotiation successful
0809 22:56:11.529996 (+ 393us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":525,"thread_start_us":345,"threads_started":1}
I20260809 22:56:11.531680 7152 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:56:11.614919 15814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskItYPJR/build/asan/bin/kudu with pid 6924
I20260809 22:56:11.638788 15814 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskItYPJR/build/asan/bin/kudu pid 7163
I20260809 22:56:11.656220 7619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:56:11.639712 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36390 (local address 127.15.113.130:39003)
0809 22:56:11.640202 (+ 490us) server_negotiation.cc:206] Beginning negotiation
0809 22:56:11.640216 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0809 22:56:11.640256 (+ 40us) server_negotiation.cc:407] Connection header received
0809 22:56:11.641811 (+ 1555us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:56:11.641821 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:56:11.642272 (+ 451us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:56:11.642607 (+ 335us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:56:11.644102 (+ 1495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:56:11.645474 (+ 1372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:56:11.646737 (+ 1263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:56:11.647415 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:56:11.647590 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:56:11.650244 (+ 2654us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:56:11.650252 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:56:11.650268 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:56:11.650306 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:56:11.653010 (+ 2704us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:56:11.653656 (+ 646us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:56:11.653663 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:56:11.653669 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:56:11.653846 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:56:11.654226 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:56:11.654232 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:56:11.654235 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:56:11.654755 (+ 520us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:56:11.655046 (+ 291us) server_negotiation.cc:1035] Waiting for connection context
0809 22:56:11.655518 (+ 472us) server_negotiation.cc:299] Negotiation successful
0809 22:56:11.655934 (+ 416us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":336,"thread_start_us":216,"threads_started":1}
I20260809 22:56:11.657778 7338 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:56:11.740042 15814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskItYPJR/build/asan/bin/kudu with pid 7163
I20260809 22:56:11.766762 15814 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskItYPJR/build/asan/bin/kudu pid 7348
I20260809 22:56:11.783721 7621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:56:11.767795 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43070 (local address 127.15.113.131:36817)
0809 22:56:11.768467 (+ 672us) server_negotiation.cc:206] Beginning negotiation
0809 22:56:11.768485 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0809 22:56:11.768528 (+ 43us) server_negotiation.cc:407] Connection header received
0809 22:56:11.770202 (+ 1674us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:56:11.770212 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:56:11.770645 (+ 433us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:56:11.770927 (+ 282us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:56:11.772221 (+ 1294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:56:11.773590 (+ 1369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:56:11.774749 (+ 1159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:56:11.775399 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:56:11.775525 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:56:11.778029 (+ 2504us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:56:11.778037 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:56:11.778051 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:56:11.778084 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:56:11.780862 (+ 2778us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:56:11.781528 (+ 666us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:56:11.781535 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:56:11.781542 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:56:11.781731 (+ 189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:56:11.782027 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:56:11.782030 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:56:11.782031 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:56:11.782558 (+ 527us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:56:11.782807 (+ 249us) server_negotiation.cc:1035] Waiting for connection context
0809 22:56:11.783092 (+ 285us) server_negotiation.cc:299] Negotiation successful
0809 22:56:11.783415 (+ 323us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":481,"thread_start_us":297,"threads_started":1}
I20260809 22:56:11.785094 7521 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:56:11.867367 15814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskItYPJR/build/asan/bin/kudu with pid 7348
2026-08-09T22:56:11Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (24352 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315935617919-15814-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 22:56:11 dist-test-slave-ncp9 krb5kdc[7625](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 22:56:11 dist-test-slave-ncp9 krb5kdc[7625](info): set up 2 sockets
Aug 09 22:56:11 dist-test-slave-ncp9 krb5kdc[7625](info): commencing operation
krb5kdc: starting...
W20260809 22:56:13.116998 15814 mini_kdc.cc:121] Time spent starting KDC: real 1.202s user 0.002s sys 0.040s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315935617919-15814-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315935617919-15814-0/krb5kdc/test-user.keytab.
Aug 09 22:56:13 dist-test-slave-ncp9 krb5kdc[7625](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316173, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T22:56:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T22:56:13Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.113.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.113.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.113.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.190.keytab.
Entry for principal kudu/127.15.113.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.190.keytab.
I20260809 22:56:13.389329 15814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskItYPJR/build/asan/bin/kudu
/tmp/dist-test-taskItYPJR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315935617919-15814-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315935617919-15814-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315935617919-15814-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315935617919-15814-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.113.190:39625
--webserver_interface=127.15.113.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.190.keytab
--principal=kudu/127.15.113.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.113.148:46195
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.113.190:39625
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315935617919-15814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315935617919-15814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315935617919-15814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:56:13.738240 7641 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:56:13.738713 7641 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:56:13.738879 7641 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:56:13.748025 7641 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260809 22:56:13.748106 7641 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260809 22:56:13.748149 7641 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260809 22:56:13.748198 7641 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 22:56:13.748250 7641 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:56:13.748315 7641 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:56:13.748358 7641 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 22:56:13.748407 7641 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 22:56:13.758232 7641 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.113.148:46195
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315935617919-15814-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315935617919-15814-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.15.113.190:39625
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.190.keytab
--principal=kudu/127.15.113.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.113.190:39625
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315935617919-15814-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.113.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315935617919-15814-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:56:13.759426 7641 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:56:13.760673 7641 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:56:13.771800 7650 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:56:13.771778 7652 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:56:13.771971 7655 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:56:13.772465 7641 server_base.cc:1034] running on GCE node
I20260809 22:56:13.773293 7641 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:56:13.774561 7641 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:56:13.775774 7641 hybrid_clock.cc:648] HybridClock initialized: now 1786316173775720 us; error 33 us; skew 500 ppm
I20260809 22:56:13.776085 7641 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:56:13 dist-test-slave-ncp9 krb5kdc[7625](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316173, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.113.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:56:13.779081 7641 init.cc:377] Logged in from keytab as kudu/127.15.113.190@KRBTEST.COM (short username kudu)
I20260809 22:56:13.780403 7641 webserver.cc:460] Webserver started at http://127.15.113.190:36581/ using document root <none> and password file <none>
I20260809 22:56:13.781075 7641 fs_manager.cc:359] Metadata directory not provided
I20260809 22:56:13.781165 7641 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:56:13.781425 7641 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:56:13.784610 7641 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315935617919-15814-0/minicluster-data/master-0/data/instance:
uuid: "0f3abae906c2437aaf43e0bef2ad018c"
format_stamp: "Formatted at 2026-08-09 22:56:13 on dist-test-slave-ncp9"
server_key: "38c512142111980ce2d84baa62069423"
server_key_iv: "bd6328eedf7756daae78341d56af6aff"
server_key_version: "encryptionkey@0"
I20260809 22:56:13.785495 7641 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315935617919-15814-0/minicluster-data/master-0/wal/instance:
uuid: "0f3abae906c2437aaf43e0bef2ad018c"
format_stamp: "Formatted at 2026-08-09 22:56:13 on dist-test-slave-ncp9"
server_key: "38c512142111980ce2d84baa62069423"
server_key_iv: "bd6328eedf7756daae78341d56af6aff"
server_key_version: "encryptionkey@0"
I20260809 22:56:13.790241 7641 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.003s
I20260809 22:56:13.793385 7667 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:56:13.794574 7641 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.002s
I20260809 22:56:13.794731 7641 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315935617919-15814-0/minicluster-data/master-0/data,/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315935617919-15814-0/minicluster-data/master-0/wal
uuid: "0f3abae906c2437aaf43e0bef2ad018c"
format_stamp: "Formatted at 2026-08-09 22:56:13 on dist-test-slave-ncp9"
server_key: "38c512142111980ce2d84baa62069423"
server_key_iv: "bd6328eedf7756daae78341d56af6aff"
server_key_version: "encryptionkey@0"
I20260809 22:56:13.794915 7641 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315935617919-15814-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315935617919-15814-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315935617919-15814-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:56:13.807519 7641 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:56:13.811055 7641 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:56:13.811296 7641 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:56:13.828383 7641 rpc_server.cc:307] RPC server started. Bound to: 127.15.113.190:39625
I20260809 22:56:13.828390 7739 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.113.190:39625 every 8 connection(s)
I20260809 22:56:13.829656 7641 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315935617919-15814-0/minicluster-data/master-0/data/info.pb
I20260809 22:56:13.833244 7740 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:56:13.833948 15814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskItYPJR/build/asan/bin/kudu as pid 7641
I20260809 22:56:13.834175 15814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315935617919-15814-0/minicluster-data/master-0/wal/instance
I20260809 22:56:13.834816 15814 external_mini_cluster.cc:1428] Setting key 12ef383e0b3bb226c8f26180482cbe09
I20260809 22:56:13.846413 7740 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0f3abae906c2437aaf43e0bef2ad018c: Bootstrap starting.
Aug 09 22:56:13 dist-test-slave-ncp9 krb5kdc[7625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316173, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.113.190@KRBTEST.COM
I20260809 22:56:13.849898 7740 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0f3abae906c2437aaf43e0bef2ad018c: Neither blocks nor log segments found. Creating new log.
I20260809 22:56:13.850931 7740 log.cc:824] T 00000000000000000000000000000000 P 0f3abae906c2437aaf43e0bef2ad018c: Log is configured to *not* fsync() on all Append() calls
I20260809 22:56:13.853582 7740 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0f3abae906c2437aaf43e0bef2ad018c: No bootstrap required, opened a new log
I20260809 22:56:13.856873 7743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:56:13.836682 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57724 (local address 127.15.113.190:39625)
0809 22:56:13.837422 (+ 740us) server_negotiation.cc:206] Beginning negotiation
0809 22:56:13.837433 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 22:56:13.837472 (+ 39us) server_negotiation.cc:407] Connection header received
0809 22:56:13.839116 (+ 1644us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:56:13.839137 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:56:13.839498 (+ 361us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:56:13.839815 (+ 317us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:56:13.841538 (+ 1723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:56:13.842844 (+ 1306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:56:13.844712 (+ 1868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:56:13.845403 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:56:13.845522 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:56:13.850167 (+ 4645us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:56:13.850174 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:56:13.850187 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:56:13.850229 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:56:13.853102 (+ 2873us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:56:13.853746 (+ 644us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:56:13.853753 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:56:13.853758 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:56:13.853909 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:56:13.854290 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:56:13.854294 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:56:13.854296 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:56:13.854794 (+ 498us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:56:13.855075 (+ 281us) server_negotiation.cc:1035] Waiting for connection context
0809 22:56:13.855525 (+ 450us) server_negotiation.cc:299] Negotiation successful
0809 22:56:13.855924 (+ 399us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":534,"thread_start_us":348,"threads_started":1}
I20260809 22:56:13.861299 7740 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0f3abae906c2437aaf43e0bef2ad018c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f3abae906c2437aaf43e0bef2ad018c" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 39625 } }
I20260809 22:56:13.861650 7740 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0f3abae906c2437aaf43e0bef2ad018c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:56:13.861754 7740 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0f3abae906c2437aaf43e0bef2ad018c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0f3abae906c2437aaf43e0bef2ad018c, State: Initialized, Role: FOLLOWER
I20260809 22:56:13.862252 7740 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0f3abae906c2437aaf43e0bef2ad018c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f3abae906c2437aaf43e0bef2ad018c" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 39625 } }
I20260809 22:56:13.862454 7740 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0f3abae906c2437aaf43e0bef2ad018c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 22:56:13.862546 7740 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0f3abae906c2437aaf43e0bef2ad018c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 22:56:13.862675 7740 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0f3abae906c2437aaf43e0bef2ad018c [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:56:13.864637 7740 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0f3abae906c2437aaf43e0bef2ad018c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f3abae906c2437aaf43e0bef2ad018c" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 39625 } }
I20260809 22:56:13.865115 7740 leader_election.cc:302] T 00000000000000000000000000000000 P 0f3abae906c2437aaf43e0bef2ad018c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0f3abae906c2437aaf43e0bef2ad018c; no voters:
I20260809 22:56:13.865626 7740 leader_election.cc:288] T 00000000000000000000000000000000 P 0f3abae906c2437aaf43e0bef2ad018c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 22:56:13.865752 7745 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0f3abae906c2437aaf43e0bef2ad018c [term 1 FOLLOWER]: Leader election won for term 1
I20260809 22:56:13.866420 7745 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0f3abae906c2437aaf43e0bef2ad018c [term 1 LEADER]: Becoming Leader. State: Replica: 0f3abae906c2437aaf43e0bef2ad018c, State: Running, Role: LEADER
I20260809 22:56:13.866879 7745 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0f3abae906c2437aaf43e0bef2ad018c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f3abae906c2437aaf43e0bef2ad018c" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 39625 } }
I20260809 22:56:13.867511 7740 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0f3abae906c2437aaf43e0bef2ad018c [sys.catalog]: configured and running, proceeding with master startup.
I20260809 22:56:13.869555 7747 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0f3abae906c2437aaf43e0bef2ad018c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0f3abae906c2437aaf43e0bef2ad018c. Latest consensus state: current_term: 1 leader_uuid: "0f3abae906c2437aaf43e0bef2ad018c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f3abae906c2437aaf43e0bef2ad018c" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 39625 } } }
I20260809 22:56:13.869916 7747 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0f3abae906c2437aaf43e0bef2ad018c [sys.catalog]: This master's current role is: LEADER
I20260809 22:56:13.870383 7746 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0f3abae906c2437aaf43e0bef2ad018c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0f3abae906c2437aaf43e0bef2ad018c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f3abae906c2437aaf43e0bef2ad018c" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 39625 } } }
I20260809 22:56:13.870925 7746 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0f3abae906c2437aaf43e0bef2ad018c [sys.catalog]: This master's current role is: LEADER
I20260809 22:56:13.871719 7756 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 22:56:13.876435 7756 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 22:56:13.885733 7756 catalog_manager.cc:1347] Generated new cluster ID: 0589e66c0b8b476bb039a06f3f16b159
I20260809 22:56:13.885842 7756 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 22:56:13.903569 7756 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 22:56:13.904729 7756 catalog_manager.cc:1504] Loading token signing keys...
I20260809 22:56:13.917354 7756 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0f3abae906c2437aaf43e0bef2ad018c: Generated new TSK 0
I20260809 22:56:13.918388 7756 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.113.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.113.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.113.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.129.keytab.
Entry for principal kudu/127.15.113.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.129.keytab.
I20260809 22:56:13.984128 15814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskItYPJR/build/asan/bin/kudu
/tmp/dist-test-taskItYPJR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315935617919-15814-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315935617919-15814-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315935617919-15814-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315935617919-15814-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.113.129:0
--local_ip_for_outbound_sockets=127.15.113.129
--webserver_interface=127.15.113.129
--webserver_port=0
--tserver_master_addrs=127.15.113.190:39625
--keytab_file=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.129.keytab
--principal=kudu/127.15.113.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.113.148:46195
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315935617919-15814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315935617919-15814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315935617919-15814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:56:14.319833 7775 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:56:14.320115 7775 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:56:14.320252 7775 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:56:14.327281 7775 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:56:14.327389 7775 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:56:14.327546 7775 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.113.129
I20260809 22:56:14.336748 7775 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.113.148:46195
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315935617919-15814-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315935617919-15814-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.129.keytab
--principal=kudu/127.15.113.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.113.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315935617919-15814-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.113.129
--webserver_port=0
--tserver_master_addrs=127.15.113.190:39625
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.113.129
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315935617919-15814-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:56:14.337930 7775 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:56:14.339184 7775 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:56:14.351758 7784 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:56:14.351714 7786 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:56:14.351976 7783 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:56:14.352833 7775 server_base.cc:1034] running on GCE node
I20260809 22:56:14.353451 7775 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:56:14.354321 7775 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:56:14.355556 7775 hybrid_clock.cc:648] HybridClock initialized: now 1786316174355496 us; error 43 us; skew 500 ppm
I20260809 22:56:14.355887 7775 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:56:14 dist-test-slave-ncp9 krb5kdc[7625](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316174, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.113.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:56:14.358913 7775 init.cc:377] Logged in from keytab as kudu/127.15.113.129@KRBTEST.COM (short username kudu)
I20260809 22:56:14.360342 7775 webserver.cc:460] Webserver started at http://127.15.113.129:34871/ using document root <none> and password file <none>
I20260809 22:56:14.361033 7775 fs_manager.cc:359] Metadata directory not provided
I20260809 22:56:14.361156 7775 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:56:14.361435 7775 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:56:14.364255 7775 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315935617919-15814-0/minicluster-data/ts-0/data/instance:
uuid: "c988eacb729c44a0bdd072dc42707c58"
format_stamp: "Formatted at 2026-08-09 22:56:14 on dist-test-slave-ncp9"
server_key: "6791de00355b98f99ffb6e181829269d"
server_key_iv: "4880d044c42044fe3964d38b4a81c2dc"
server_key_version: "encryptionkey@0"
I20260809 22:56:14.365046 7775 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315935617919-15814-0/minicluster-data/ts-0/wal/instance:
uuid: "c988eacb729c44a0bdd072dc42707c58"
format_stamp: "Formatted at 2026-08-09 22:56:14 on dist-test-slave-ncp9"
server_key: "6791de00355b98f99ffb6e181829269d"
server_key_iv: "4880d044c42044fe3964d38b4a81c2dc"
server_key_version: "encryptionkey@0"
I20260809 22:56:14.369957 7775 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260809 22:56:14.373255 7796 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:56:14.374492 7775 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 22:56:14.374639 7775 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315935617919-15814-0/minicluster-data/ts-0/data,/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315935617919-15814-0/minicluster-data/ts-0/wal
uuid: "c988eacb729c44a0bdd072dc42707c58"
format_stamp: "Formatted at 2026-08-09 22:56:14 on dist-test-slave-ncp9"
server_key: "6791de00355b98f99ffb6e181829269d"
server_key_iv: "4880d044c42044fe3964d38b4a81c2dc"
server_key_version: "encryptionkey@0"
I20260809 22:56:14.374819 7775 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315935617919-15814-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315935617919-15814-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315935617919-15814-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:56:14.393688 7775 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:56:14.397243 7775 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:56:14.397492 7775 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:56:14.398397 7775 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:56:14.399641 7775 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:56:14.399727 7775 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:56:14.399844 7775 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:56:14.399906 7775 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:56:14.429939 7775 rpc_server.cc:307] RPC server started. Bound to: 127.15.113.129:40825
I20260809 22:56:14.429993 7954 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.113.129:40825 every 8 connection(s)
I20260809 22:56:14.431142 7775 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315935617919-15814-0/minicluster-data/ts-0/data/info.pb
I20260809 22:56:14.431857 15814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskItYPJR/build/asan/bin/kudu as pid 7775
I20260809 22:56:14.432097 15814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315935617919-15814-0/minicluster-data/ts-0/wal/instance
I20260809 22:56:14.432845 15814 external_mini_cluster.cc:1428] Setting key 4dbbf42a1f71b2d3b5d1443232030cb7
Aug 09 22:56:14 dist-test-slave-ncp9 krb5kdc[7625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316174, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.113.129@KRBTEST.COM for kudu/127.15.113.190@KRBTEST.COM
I20260809 22:56:14.452869 7958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:56:14.433830 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.113.129:53115 (local address 127.15.113.190:39625)
0809 22:56:14.434239 (+ 409us) server_negotiation.cc:206] Beginning negotiation
0809 22:56:14.434244 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:56:14.437179 (+ 2935us) server_negotiation.cc:407] Connection header received
0809 22:56:14.438985 (+ 1806us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:56:14.438990 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:56:14.439068 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:56:14.439163 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:56:14.440679 (+ 1516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:56:14.441445 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:56:14.443030 (+ 1585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:56:14.443426 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:56:14.443532 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:56:14.447077 (+ 3545us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:56:14.447082 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:56:14.447086 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:56:14.447118 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:56:14.450018 (+ 2900us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:56:14.450686 (+ 668us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:56:14.450692 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:56:14.450696 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:56:14.450833 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:56:14.451215 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:56:14.451220 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:56:14.451223 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:56:14.451565 (+ 342us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:56:14.451704 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0809 22:56:14.452309 (+ 605us) server_negotiation.cc:299] Negotiation successful
0809 22:56:14.452541 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":294,"thread_start_us":195,"threads_started":1}
I20260809 22:56:14.453590 7960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:56:14.434453 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:39625 (local address 127.15.113.129:53115)
0809 22:56:14.437034 (+ 2581us) negotiation.cc:107] Waiting for socket to connect
0809 22:56:14.437074 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0809 22:56:14.438732 (+ 1658us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:56:14.439417 (+ 685us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:56:14.439425 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:56:14.439938 (+ 513us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:56:14.440506 (+ 568us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:56:14.440524 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:56:14.441593 (+ 1069us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:56:14.441597 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:56:14.442896 (+ 1299us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:56:14.442909 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:56:14.443642 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:56:14.443649 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:56:14.443864 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:56:14.444458 (+ 594us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:56:14.444484 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:56:14.446870 (+ 2386us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:56:14.450155 (+ 3285us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:56:14.450162 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:56:14.450173 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:56:14.450566 (+ 393us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:56:14.450923 (+ 357us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:56:14.450926 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:56:14.450928 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:56:14.451095 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:56:14.451669 (+ 574us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:56:14.451674 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:56:14.451959 (+ 285us) client_negotiation.cc:769] Sending connection context
0809 22:56:14.452182 (+ 223us) client_negotiation.cc:240] Negotiation successful
0809 22:56:14.452532 (+ 350us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":518,"thread_start_us":315,"threads_started":1}
I20260809 22:56:14.454973 7955 heartbeater.cc:347] Connected to a master server at 127.15.113.190:39625
I20260809 22:56:14.455242 7955 heartbeater.cc:464] Registering TS with master...
I20260809 22:56:14.455928 7955 heartbeater.cc:511] Master 127.15.113.190:39625 requested a full tablet report, sending...
I20260809 22:56:14.457929 7693 ts_manager.cc:194] Registered new tserver with Master: c988eacb729c44a0bdd072dc42707c58 (127.15.113.129:40825)
I20260809 22:56:14.459339 7693 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.113.129@KRBTEST.COM'} at 127.15.113.129:53115
WARNING: no policy specified for kudu/127.15.113.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.113.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.113.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.130.keytab.
Entry for principal kudu/127.15.113.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.130.keytab.
I20260809 22:56:14.497620 15814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskItYPJR/build/asan/bin/kudu
/tmp/dist-test-taskItYPJR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315935617919-15814-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315935617919-15814-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315935617919-15814-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315935617919-15814-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.113.130:0
--local_ip_for_outbound_sockets=127.15.113.130
--webserver_interface=127.15.113.130
--webserver_port=0
--tserver_master_addrs=127.15.113.190:39625
--keytab_file=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.130.keytab
--principal=kudu/127.15.113.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.113.148:46195
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315935617919-15814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315935617919-15814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315935617919-15814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:56:14.827066 7963 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:56:14.827329 7963 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:56:14.827428 7963 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:56:14.838196 7963 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:56:14.838317 7963 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:56:14.838498 7963 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.113.130
I20260809 22:56:14.852232 7963 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.113.148:46195
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315935617919-15814-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315935617919-15814-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.130.keytab
--principal=kudu/127.15.113.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.113.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315935617919-15814-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.113.130
--webserver_port=0
--tserver_master_addrs=127.15.113.190:39625
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.113.130
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315935617919-15814-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:56:14.853384 7963 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:56:14.854588 7963 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:56:14.866837 7969 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:56:14.866879 7971 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:56:14.866796 7973 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:56:14.867753 7963 server_base.cc:1034] running on GCE node
I20260809 22:56:14.868276 7963 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:56:14.869226 7963 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:56:14.870414 7963 hybrid_clock.cc:648] HybridClock initialized: now 1786316174870372 us; error 31 us; skew 500 ppm
I20260809 22:56:14.870708 7963 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:56:14 dist-test-slave-ncp9 krb5kdc[7625](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316174, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.113.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:56:14.873631 7963 init.cc:377] Logged in from keytab as kudu/127.15.113.130@KRBTEST.COM (short username kudu)
I20260809 22:56:14.874982 7963 webserver.cc:460] Webserver started at http://127.15.113.130:45625/ using document root <none> and password file <none>
I20260809 22:56:14.875593 7963 fs_manager.cc:359] Metadata directory not provided
I20260809 22:56:14.875705 7963 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:56:14.875990 7963 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:56:14.878834 7963 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315935617919-15814-0/minicluster-data/ts-1/data/instance:
uuid: "0022f56469854dd5b9783b4e88420ef5"
format_stamp: "Formatted at 2026-08-09 22:56:14 on dist-test-slave-ncp9"
server_key: "9a840933e2926fdf96995a968373eb05"
server_key_iv: "b62d759303acd0b00a4efd1623e5c932"
server_key_version: "encryptionkey@0"
I20260809 22:56:14.879581 7963 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315935617919-15814-0/minicluster-data/ts-1/wal/instance:
uuid: "0022f56469854dd5b9783b4e88420ef5"
format_stamp: "Formatted at 2026-08-09 22:56:14 on dist-test-slave-ncp9"
server_key: "9a840933e2926fdf96995a968373eb05"
server_key_iv: "b62d759303acd0b00a4efd1623e5c932"
server_key_version: "encryptionkey@0"
I20260809 22:56:14.884380 7963 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260809 22:56:14.887676 7987 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:56:14.889115 7963 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 22:56:14.889282 7963 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315935617919-15814-0/minicluster-data/ts-1/data,/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315935617919-15814-0/minicluster-data/ts-1/wal
uuid: "0022f56469854dd5b9783b4e88420ef5"
format_stamp: "Formatted at 2026-08-09 22:56:14 on dist-test-slave-ncp9"
server_key: "9a840933e2926fdf96995a968373eb05"
server_key_iv: "b62d759303acd0b00a4efd1623e5c932"
server_key_version: "encryptionkey@0"
I20260809 22:56:14.889489 7963 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315935617919-15814-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315935617919-15814-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315935617919-15814-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:56:14.903393 7963 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:56:14.906998 7963 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:56:14.907263 7963 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:56:14.908118 7963 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:56:14.909403 7963 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:56:14.909476 7963 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:56:14.909588 7963 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:56:14.909649 7963 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:56:14.939747 7963 rpc_server.cc:307] RPC server started. Bound to: 127.15.113.130:33009
I20260809 22:56:14.939797 8149 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.113.130:33009 every 8 connection(s)
I20260809 22:56:14.940992 7963 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315935617919-15814-0/minicluster-data/ts-1/data/info.pb
I20260809 22:56:14.944849 15814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskItYPJR/build/asan/bin/kudu as pid 7963
I20260809 22:56:14.945071 15814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315935617919-15814-0/minicluster-data/ts-1/wal/instance
I20260809 22:56:14.945554 15814 external_mini_cluster.cc:1428] Setting key b0ae2319c8b845f5bcb370bca959c12f
Aug 09 22:56:14 dist-test-slave-ncp9 krb5kdc[7625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316174, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.113.130@KRBTEST.COM for kudu/127.15.113.190@KRBTEST.COM
I20260809 22:56:14.961258 7958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:56:14.943431 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.113.130:45395 (local address 127.15.113.190:39625)
0809 22:56:14.943601 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0809 22:56:14.943605 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 22:56:14.944790 (+ 1185us) server_negotiation.cc:407] Connection header received
0809 22:56:14.946903 (+ 2113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:56:14.946909 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:56:14.946988 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:56:14.947096 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:56:14.948545 (+ 1449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:56:14.949556 (+ 1011us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:56:14.951035 (+ 1479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:56:14.951616 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:56:14.951736 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:56:14.955455 (+ 3719us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:56:14.955461 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:56:14.955465 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:56:14.955502 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:56:14.958204 (+ 2702us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:56:14.958988 (+ 784us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:56:14.958992 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:56:14.958995 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:56:14.959121 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:56:14.959541 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:56:14.959545 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:56:14.959548 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:56:14.959813 (+ 265us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:56:14.959933 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0809 22:56:14.960665 (+ 732us) server_negotiation.cc:299] Negotiation successful
0809 22:56:14.960883 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260809 22:56:14.962486 8156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:56:14.943949 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:39625 (local address 127.15.113.130:45395)
0809 22:56:14.944639 (+ 690us) negotiation.cc:107] Waiting for socket to connect
0809 22:56:14.944679 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0809 22:56:14.946575 (+ 1896us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:56:14.947329 (+ 754us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:56:14.947339 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:56:14.947836 (+ 497us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:56:14.948377 (+ 541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:56:14.948395 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:56:14.949684 (+ 1289us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:56:14.949688 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:56:14.950895 (+ 1207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:56:14.950907 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:56:14.951795 (+ 888us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:56:14.951799 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:56:14.951987 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:56:14.952597 (+ 610us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:56:14.952618 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:56:14.955233 (+ 2615us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:56:14.958353 (+ 3120us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:56:14.958361 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:56:14.958376 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:56:14.958894 (+ 518us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:56:14.959222 (+ 328us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:56:14.959226 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:56:14.959229 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:56:14.959456 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:56:14.959931 (+ 475us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:56:14.959938 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:56:14.960329 (+ 391us) client_negotiation.cc:769] Sending connection context
0809 22:56:14.960600 (+ 271us) client_negotiation.cc:240] Negotiation successful
0809 22:56:14.961171 (+ 571us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":469,"thread_start_us":274,"threads_started":1}
I20260809 22:56:14.963755 8151 heartbeater.cc:347] Connected to a master server at 127.15.113.190:39625
I20260809 22:56:14.964031 8151 heartbeater.cc:464] Registering TS with master...
I20260809 22:56:14.964720 8151 heartbeater.cc:511] Master 127.15.113.190:39625 requested a full tablet report, sending...
I20260809 22:56:14.966118 7693 ts_manager.cc:194] Registered new tserver with Master: 0022f56469854dd5b9783b4e88420ef5 (127.15.113.130:33009)
I20260809 22:56:14.966920 7693 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.113.130@KRBTEST.COM'} at 127.15.113.130:45395
WARNING: no policy specified for kudu/127.15.113.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.113.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.113.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.131.keytab.
Entry for principal kudu/127.15.113.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.131.keytab.
I20260809 22:56:15.008116 15814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskItYPJR/build/asan/bin/kudu
/tmp/dist-test-taskItYPJR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315935617919-15814-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315935617919-15814-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315935617919-15814-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315935617919-15814-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.113.131:0
--local_ip_for_outbound_sockets=127.15.113.131
--webserver_interface=127.15.113.131
--webserver_port=0
--tserver_master_addrs=127.15.113.190:39625
--keytab_file=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.131.keytab
--principal=kudu/127.15.113.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.113.148:46195
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315935617919-15814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315935617919-15814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315935617919-15814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:56:15.329082 8159 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:56:15.329350 8159 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:56:15.329488 8159 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:56:15.336439 8159 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:56:15.336540 8159 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:56:15.336687 8159 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.113.131
I20260809 22:56:15.345866 8159 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.113.148:46195
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315935617919-15814-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315935617919-15814-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.131.keytab
--principal=kudu/127.15.113.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.113.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315935617919-15814-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.113.131
--webserver_port=0
--tserver_master_addrs=127.15.113.190:39625
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.113.131
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315935617919-15814-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:56:15.347000 8159 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:56:15.348266 8159 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:56:15.360864 8169 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:56:15.361145 8173 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:56:15.361168 8170 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:56:15.361574 8159 server_base.cc:1034] running on GCE node
I20260809 22:56:15.362108 8159 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:56:15.363068 8159 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:56:15.364246 8159 hybrid_clock.cc:648] HybridClock initialized: now 1786316175364196 us; error 48 us; skew 500 ppm
I20260809 22:56:15.364496 8159 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:56:15 dist-test-slave-ncp9 krb5kdc[7625](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316175, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.113.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:56:15.367377 8159 init.cc:377] Logged in from keytab as kudu/127.15.113.131@KRBTEST.COM (short username kudu)
I20260809 22:56:15.368752 8159 webserver.cc:460] Webserver started at http://127.15.113.131:33209/ using document root <none> and password file <none>
I20260809 22:56:15.369419 8159 fs_manager.cc:359] Metadata directory not provided
I20260809 22:56:15.369534 8159 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:56:15.369817 8159 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:56:15.372545 8159 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315935617919-15814-0/minicluster-data/ts-2/data/instance:
uuid: "a8594d32d7724c3b831d36b48c70ba60"
format_stamp: "Formatted at 2026-08-09 22:56:15 on dist-test-slave-ncp9"
server_key: "a9b5faab3611544f4151bd0358783ef9"
server_key_iv: "a82a94e30a001d1c2576a2c51c70eeee"
server_key_version: "encryptionkey@0"
I20260809 22:56:15.373313 8159 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315935617919-15814-0/minicluster-data/ts-2/wal/instance:
uuid: "a8594d32d7724c3b831d36b48c70ba60"
format_stamp: "Formatted at 2026-08-09 22:56:15 on dist-test-slave-ncp9"
server_key: "a9b5faab3611544f4151bd0358783ef9"
server_key_iv: "a82a94e30a001d1c2576a2c51c70eeee"
server_key_version: "encryptionkey@0"
I20260809 22:56:15.378121 8159 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.004s
I20260809 22:56:15.381315 8181 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:56:15.382539 8159 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260809 22:56:15.382676 8159 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315935617919-15814-0/minicluster-data/ts-2/data,/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315935617919-15814-0/minicluster-data/ts-2/wal
uuid: "a8594d32d7724c3b831d36b48c70ba60"
format_stamp: "Formatted at 2026-08-09 22:56:15 on dist-test-slave-ncp9"
server_key: "a9b5faab3611544f4151bd0358783ef9"
server_key_iv: "a82a94e30a001d1c2576a2c51c70eeee"
server_key_version: "encryptionkey@0"
I20260809 22:56:15.382838 8159 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315935617919-15814-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315935617919-15814-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315935617919-15814-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:56:15.402868 8159 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:56:15.406420 8159 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:56:15.406663 8159 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:56:15.407531 8159 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:56:15.408767 8159 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:56:15.408847 8159 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:56:15.408982 8159 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:56:15.409037 8159 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:56:15.442278 8159 rpc_server.cc:307] RPC server started. Bound to: 127.15.113.131:46709
I20260809 22:56:15.442317 8342 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.113.131:46709 every 8 connection(s)
I20260809 22:56:15.443462 8159 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315935617919-15814-0/minicluster-data/ts-2/data/info.pb
I20260809 22:56:15.452143 15814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskItYPJR/build/asan/bin/kudu as pid 8159
I20260809 22:56:15.452333 15814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315935617919-15814-0/minicluster-data/ts-2/wal/instance
I20260809 22:56:15.452769 15814 external_mini_cluster.cc:1428] Setting key 839fd0811c3b7e656b7b9729725214d3
Aug 09 22:56:15 dist-test-slave-ncp9 krb5kdc[7625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316175, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.113.131@KRBTEST.COM for kudu/127.15.113.190@KRBTEST.COM
I20260809 22:56:15.462572 7955 heartbeater.cc:503] Master 127.15.113.190:39625 was elected leader, sending a full tablet report...
I20260809 22:56:15.464802 7958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:56:15.447330 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.113.131:38139 (local address 127.15.113.190:39625)
0809 22:56:15.448285 (+ 955us) server_negotiation.cc:206] Beginning negotiation
0809 22:56:15.448292 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:56:15.448730 (+ 438us) server_negotiation.cc:407] Connection header received
0809 22:56:15.451134 (+ 2404us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:56:15.451141 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:56:15.451230 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:56:15.451324 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:56:15.452970 (+ 1646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:56:15.453805 (+ 835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:56:15.455708 (+ 1903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:56:15.456105 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:56:15.456211 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:56:15.459365 (+ 3154us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:56:15.459369 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:56:15.459373 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:56:15.459404 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:56:15.461776 (+ 2372us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:56:15.462493 (+ 717us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:56:15.462498 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:56:15.462500 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:56:15.462639 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:56:15.463093 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:56:15.463098 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:56:15.463102 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:56:15.463419 (+ 317us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:56:15.463564 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0809 22:56:15.464295 (+ 731us) server_negotiation.cc:299] Negotiation successful
0809 22:56:15.464505 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":810}
I20260809 22:56:15.466709 8346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:56:15.447818 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:39625 (local address 127.15.113.131:38139)
0809 22:56:15.448570 (+ 752us) negotiation.cc:107] Waiting for socket to connect
0809 22:56:15.448605 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0809 22:56:15.450810 (+ 2205us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:56:15.451714 (+ 904us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:56:15.451723 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:56:15.452241 (+ 518us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:56:15.452777 (+ 536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:56:15.452796 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:56:15.453985 (+ 1189us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:56:15.453991 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:56:15.455575 (+ 1584us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:56:15.455589 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:56:15.456294 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:56:15.456299 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:56:15.456484 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:56:15.457108 (+ 624us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:56:15.457129 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:56:15.459186 (+ 2057us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:56:15.461895 (+ 2709us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:56:15.461901 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:56:15.461912 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:56:15.462378 (+ 466us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:56:15.462745 (+ 367us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:56:15.462749 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:56:15.462751 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:56:15.462978 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:56:15.463552 (+ 574us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:56:15.463560 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:56:15.463852 (+ 292us) client_negotiation.cc:769] Sending connection context
0809 22:56:15.465000 (+ 1148us) client_negotiation.cc:240] Negotiation successful
0809 22:56:15.465440 (+ 440us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":502,"thread_start_us":373,"threads_started":1}
I20260809 22:56:15.468068 8343 heartbeater.cc:347] Connected to a master server at 127.15.113.190:39625
I20260809 22:56:15.468336 8343 heartbeater.cc:464] Registering TS with master...
I20260809 22:56:15.469027 8343 heartbeater.cc:511] Master 127.15.113.190:39625 requested a full tablet report, sending...
I20260809 22:56:15.470297 7693 ts_manager.cc:194] Registered new tserver with Master: a8594d32d7724c3b831d36b48c70ba60 (127.15.113.131:46709)
I20260809 22:56:15.471035 7693 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.113.131@KRBTEST.COM'} at 127.15.113.131:38139
I20260809 22:56:15.479710 15814 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 22:56:15.496600 7958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:56:15.483999 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57736 (local address 127.15.113.190:39625)
0809 22:56:15.484160 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0809 22:56:15.484166 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:56:15.484543 (+ 377us) server_negotiation.cc:407] Connection header received
0809 22:56:15.484699 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:56:15.484705 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:56:15.484779 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:56:15.484857 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:56:15.486404 (+ 1547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:56:15.487135 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:56:15.488511 (+ 1376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:56:15.488968 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:56:15.489078 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:56:15.491373 (+ 2295us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:56:15.491378 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:56:15.491381 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:56:15.491410 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:56:15.493944 (+ 2534us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:56:15.494718 (+ 774us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:56:15.494724 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:56:15.494728 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:56:15.494856 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:56:15.495244 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:56:15.495249 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:56:15.495253 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:56:15.495561 (+ 308us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:56:15.495800 (+ 239us) server_negotiation.cc:1035] Waiting for connection context
0809 22:56:15.496113 (+ 313us) server_negotiation.cc:299] Negotiation successful
0809 22:56:15.496316 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260809 22:56:15.500340 7693 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57736:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 22:56:15.507010 7693 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 22:56:15.538568 8360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:56:15.518304 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36496 (local address 127.15.113.129:40825)
0809 22:56:15.530580 (+ 12276us) server_negotiation.cc:206] Beginning negotiation
0809 22:56:15.530589 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 22:56:15.530619 (+ 30us) server_negotiation.cc:407] Connection header received
0809 22:56:15.530726 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:56:15.530734 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:56:15.530976 (+ 242us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:56:15.531187 (+ 211us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:56:15.531954 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:56:15.533169 (+ 1215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:56:15.535996 (+ 2827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:56:15.537423 (+ 1427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:56:15.537619 (+ 196us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:56:15.537804 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0809 22:56:15.538088 (+ 284us) server_negotiation.cc:299] Negotiation successful
0809 22:56:15.538263 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12106,"thread_start_us":13054,"threads_started":1}
I20260809 22:56:15.538690 8358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:56:15.518234 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.129:40825 (local address 127.0.0.1:36496)
0809 22:56:15.529793 (+ 11559us) negotiation.cc:107] Waiting for socket to connect
0809 22:56:15.529838 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0809 22:56:15.530027 (+ 189us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:56:15.531211 (+ 1184us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:56:15.531220 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:56:15.531254 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:56:15.531692 (+ 438us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:56:15.531712 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:56:15.533344 (+ 1632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:56:15.533349 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:56:15.535813 (+ 2464us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:56:15.535829 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:56:15.537629 (+ 1800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:56:15.537635 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:56:15.537870 (+ 235us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:56:15.537907 (+ 37us) client_negotiation.cc:769] Sending connection context
0809 22:56:15.538030 (+ 123us) client_negotiation.cc:240] Negotiation successful
0809 22:56:15.538224 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11277,"mutex_wait_us":24,"thread_start_us":13754,"threads_started":1}
I20260809 22:56:15.541364 8359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:56:15.518256 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.131:46709 (local address 127.0.0.1:55368)
0809 22:56:15.531922 (+ 13666us) negotiation.cc:107] Waiting for socket to connect
0809 22:56:15.531940 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 22:56:15.532048 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:56:15.534229 (+ 2181us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:56:15.534234 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:56:15.534257 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:56:15.534745 (+ 488us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:56:15.534755 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:56:15.536109 (+ 1354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:56:15.536113 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:56:15.538372 (+ 2259us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:56:15.538388 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:56:15.540502 (+ 2114us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:56:15.540509 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:56:15.540684 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:56:15.540714 (+ 30us) client_negotiation.cc:769] Sending connection context
0809 22:56:15.540807 (+ 93us) client_negotiation.cc:240] Negotiation successful
0809 22:56:15.540957 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13551,"mutex_wait_us":25,"thread_start_us":11978,"threads_started":1}
I20260809 22:56:15.541688 8370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:56:15.520074 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.130:33009 (local address 127.0.0.1:39538)
0809 22:56:15.532974 (+ 12900us) negotiation.cc:107] Waiting for socket to connect
0809 22:56:15.532991 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 22:56:15.533114 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:56:15.533951 (+ 837us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:56:15.533955 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:56:15.533983 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:56:15.534449 (+ 466us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:56:15.534462 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:56:15.536303 (+ 1841us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:56:15.536309 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:56:15.538496 (+ 2187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:56:15.538507 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:56:15.540879 (+ 2372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:56:15.540885 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:56:15.541069 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:56:15.541101 (+ 32us) client_negotiation.cc:769] Sending connection context
0809 22:56:15.541182 (+ 81us) client_negotiation.cc:240] Negotiation successful
0809 22:56:15.541308 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12748,"mutex_wait_us":4,"thread_start_us":12614,"threads_started":1}
I20260809 22:56:15.542282 8362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:56:15.518899 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55368 (local address 127.15.113.131:46709)
0809 22:56:15.533664 (+ 14765us) server_negotiation.cc:206] Beginning negotiation
0809 22:56:15.533673 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 22:56:15.533707 (+ 34us) server_negotiation.cc:407] Connection header received
0809 22:56:15.533798 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:56:15.533805 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:56:15.534028 (+ 223us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:56:15.534227 (+ 199us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:56:15.534905 (+ 678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:56:15.535971 (+ 1066us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:56:15.538582 (+ 2611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:56:15.539986 (+ 1404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:56:15.540189 (+ 203us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:56:15.540354 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0809 22:56:15.541732 (+ 1378us) server_negotiation.cc:299] Negotiation successful
0809 22:56:15.541910 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14567,"thread_start_us":14784,"threads_started":1}
I20260809 22:56:15.542405 8371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:56:15.520639 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39538 (local address 127.15.113.130:33009)
0809 22:56:15.532544 (+ 11905us) server_negotiation.cc:206] Beginning negotiation
0809 22:56:15.532552 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 22:56:15.533112 (+ 560us) server_negotiation.cc:407] Connection header received
0809 22:56:15.533337 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:56:15.533346 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:56:15.533607 (+ 261us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:56:15.533802 (+ 195us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:56:15.534664 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:56:15.536140 (+ 1476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:56:15.538693 (+ 2553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:56:15.540400 (+ 1707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:56:15.540560 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:56:15.540732 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0809 22:56:15.541950 (+ 1218us) server_negotiation.cc:299] Negotiation successful
0809 22:56:15.542119 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11740,"thread_start_us":11643,"threads_started":1}
I20260809 22:56:15.542594 7865 tablet_service.cc:1467] Processing CreateTablet for tablet a28c08be163445c9884d63b673afd5ad (DEFAULT_TABLE table=test-table [id=ab2f6751ac9f45c5b9054304e19affd2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:56:15.545289 8251 tablet_service.cc:1467] Processing CreateTablet for tablet a28c08be163445c9884d63b673afd5ad (DEFAULT_TABLE table=test-table [id=ab2f6751ac9f45c5b9054304e19affd2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:56:15.545535 8060 tablet_service.cc:1467] Processing CreateTablet for tablet a28c08be163445c9884d63b673afd5ad (DEFAULT_TABLE table=test-table [id=ab2f6751ac9f45c5b9054304e19affd2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:56:15.546691 8251 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a28c08be163445c9884d63b673afd5ad. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:56:15.546739 8060 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a28c08be163445c9884d63b673afd5ad. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:56:15.549557 7865 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a28c08be163445c9884d63b673afd5ad. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:56:15.557839 8372 tablet_bootstrap.cc:492] T a28c08be163445c9884d63b673afd5ad P a8594d32d7724c3b831d36b48c70ba60: Bootstrap starting.
I20260809 22:56:15.561632 8374 tablet_bootstrap.cc:492] T a28c08be163445c9884d63b673afd5ad P 0022f56469854dd5b9783b4e88420ef5: Bootstrap starting.
I20260809 22:56:15.562045 8372 tablet_bootstrap.cc:654] T a28c08be163445c9884d63b673afd5ad P a8594d32d7724c3b831d36b48c70ba60: Neither blocks nor log segments found. Creating new log.
I20260809 22:56:15.563486 8372 log.cc:824] T a28c08be163445c9884d63b673afd5ad P a8594d32d7724c3b831d36b48c70ba60: Log is configured to *not* fsync() on all Append() calls
I20260809 22:56:15.563616 8376 tablet_bootstrap.cc:492] T a28c08be163445c9884d63b673afd5ad P c988eacb729c44a0bdd072dc42707c58: Bootstrap starting.
I20260809 22:56:15.565057 8374 tablet_bootstrap.cc:654] T a28c08be163445c9884d63b673afd5ad P 0022f56469854dd5b9783b4e88420ef5: Neither blocks nor log segments found. Creating new log.
I20260809 22:56:15.566272 8374 log.cc:824] T a28c08be163445c9884d63b673afd5ad P 0022f56469854dd5b9783b4e88420ef5: Log is configured to *not* fsync() on all Append() calls
I20260809 22:56:15.566761 8376 tablet_bootstrap.cc:654] T a28c08be163445c9884d63b673afd5ad P c988eacb729c44a0bdd072dc42707c58: Neither blocks nor log segments found. Creating new log.
I20260809 22:56:15.567210 8372 tablet_bootstrap.cc:492] T a28c08be163445c9884d63b673afd5ad P a8594d32d7724c3b831d36b48c70ba60: No bootstrap required, opened a new log
I20260809 22:56:15.567541 8372 ts_tablet_manager.cc:1397] T a28c08be163445c9884d63b673afd5ad P a8594d32d7724c3b831d36b48c70ba60: Time spent bootstrapping tablet: real 0.010s user 0.008s sys 0.000s
I20260809 22:56:15.567983 8376 log.cc:824] T a28c08be163445c9884d63b673afd5ad P c988eacb729c44a0bdd072dc42707c58: Log is configured to *not* fsync() on all Append() calls
I20260809 22:56:15.570039 8374 tablet_bootstrap.cc:492] T a28c08be163445c9884d63b673afd5ad P 0022f56469854dd5b9783b4e88420ef5: No bootstrap required, opened a new log
I20260809 22:56:15.570304 8374 ts_tablet_manager.cc:1397] T a28c08be163445c9884d63b673afd5ad P 0022f56469854dd5b9783b4e88420ef5: Time spent bootstrapping tablet: real 0.009s user 0.000s sys 0.006s
I20260809 22:56:15.570506 8376 tablet_bootstrap.cc:492] T a28c08be163445c9884d63b673afd5ad P c988eacb729c44a0bdd072dc42707c58: No bootstrap required, opened a new log
I20260809 22:56:15.570720 8376 ts_tablet_manager.cc:1397] T a28c08be163445c9884d63b673afd5ad P c988eacb729c44a0bdd072dc42707c58: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.005s
I20260809 22:56:15.577519 8376 raft_consensus.cc:348] T a28c08be163445c9884d63b673afd5ad P c988eacb729c44a0bdd072dc42707c58 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c988eacb729c44a0bdd072dc42707c58" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 40825 } } peers { permanent_uuid: "a8594d32d7724c3b831d36b48c70ba60" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 46709 } } peers { permanent_uuid: "0022f56469854dd5b9783b4e88420ef5" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 33009 } }
I20260809 22:56:15.578058 8376 raft_consensus.cc:374] T a28c08be163445c9884d63b673afd5ad P c988eacb729c44a0bdd072dc42707c58 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:56:15.577965 8372 raft_consensus.cc:348] T a28c08be163445c9884d63b673afd5ad P a8594d32d7724c3b831d36b48c70ba60 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c988eacb729c44a0bdd072dc42707c58" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 40825 } } peers { permanent_uuid: "a8594d32d7724c3b831d36b48c70ba60" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 46709 } } peers { permanent_uuid: "0022f56469854dd5b9783b4e88420ef5" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 33009 } }
I20260809 22:56:15.578266 8376 raft_consensus.cc:729] T a28c08be163445c9884d63b673afd5ad P c988eacb729c44a0bdd072dc42707c58 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c988eacb729c44a0bdd072dc42707c58, State: Initialized, Role: FOLLOWER
I20260809 22:56:15.578491 8372 raft_consensus.cc:374] T a28c08be163445c9884d63b673afd5ad P a8594d32d7724c3b831d36b48c70ba60 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:56:15.578636 8372 raft_consensus.cc:729] T a28c08be163445c9884d63b673afd5ad P a8594d32d7724c3b831d36b48c70ba60 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a8594d32d7724c3b831d36b48c70ba60, State: Initialized, Role: FOLLOWER
I20260809 22:56:15.578926 8376 consensus_queue.cc:260] T a28c08be163445c9884d63b673afd5ad P c988eacb729c44a0bdd072dc42707c58 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c988eacb729c44a0bdd072dc42707c58" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 40825 } } peers { permanent_uuid: "a8594d32d7724c3b831d36b48c70ba60" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 46709 } } peers { permanent_uuid: "0022f56469854dd5b9783b4e88420ef5" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 33009 } }
I20260809 22:56:15.579355 8372 consensus_queue.cc:260] T a28c08be163445c9884d63b673afd5ad P a8594d32d7724c3b831d36b48c70ba60 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c988eacb729c44a0bdd072dc42707c58" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 40825 } } peers { permanent_uuid: "a8594d32d7724c3b831d36b48c70ba60" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 46709 } } peers { permanent_uuid: "0022f56469854dd5b9783b4e88420ef5" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 33009 } }
I20260809 22:56:15.580643 8376 ts_tablet_manager.cc:1428] T a28c08be163445c9884d63b673afd5ad P c988eacb729c44a0bdd072dc42707c58: Time spent starting tablet: real 0.010s user 0.005s sys 0.005s
I20260809 22:56:15.580539 8374 raft_consensus.cc:348] T a28c08be163445c9884d63b673afd5ad P 0022f56469854dd5b9783b4e88420ef5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c988eacb729c44a0bdd072dc42707c58" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 40825 } } peers { permanent_uuid: "a8594d32d7724c3b831d36b48c70ba60" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 46709 } } peers { permanent_uuid: "0022f56469854dd5b9783b4e88420ef5" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 33009 } }
I20260809 22:56:15.581084 8374 raft_consensus.cc:374] T a28c08be163445c9884d63b673afd5ad P 0022f56469854dd5b9783b4e88420ef5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:56:15.581211 8374 raft_consensus.cc:729] T a28c08be163445c9884d63b673afd5ad P 0022f56469854dd5b9783b4e88420ef5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0022f56469854dd5b9783b4e88420ef5, State: Initialized, Role: FOLLOWER
I20260809 22:56:15.581614 8343 heartbeater.cc:503] Master 127.15.113.190:39625 was elected leader, sending a full tablet report...
I20260809 22:56:15.581818 8374 consensus_queue.cc:260] T a28c08be163445c9884d63b673afd5ad P 0022f56469854dd5b9783b4e88420ef5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c988eacb729c44a0bdd072dc42707c58" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 40825 } } peers { permanent_uuid: "a8594d32d7724c3b831d36b48c70ba60" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 46709 } } peers { permanent_uuid: "0022f56469854dd5b9783b4e88420ef5" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 33009 } }
I20260809 22:56:15.582332 8372 ts_tablet_manager.cc:1428] T a28c08be163445c9884d63b673afd5ad P a8594d32d7724c3b831d36b48c70ba60: Time spent starting tablet: real 0.015s user 0.012s sys 0.003s
I20260809 22:56:15.583209 8151 heartbeater.cc:503] Master 127.15.113.190:39625 was elected leader, sending a full tablet report...
I20260809 22:56:15.583832 8374 ts_tablet_manager.cc:1428] T a28c08be163445c9884d63b673afd5ad P 0022f56469854dd5b9783b4e88420ef5: Time spent starting tablet: real 0.013s user 0.007s sys 0.006s
W20260809 22:56:15.683370 7957 tablet.cc:2369] T a28c08be163445c9884d63b673afd5ad P c988eacb729c44a0bdd072dc42707c58: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 22:56:15.692860 8155 tablet.cc:2369] T a28c08be163445c9884d63b673afd5ad P 0022f56469854dd5b9783b4e88420ef5: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 22:56:15.696324 8344 tablet.cc:2369] T a28c08be163445c9884d63b673afd5ad P a8594d32d7724c3b831d36b48c70ba60: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 22:56:15.778198 8382 raft_consensus.cc:482] T a28c08be163445c9884d63b673afd5ad P c988eacb729c44a0bdd072dc42707c58 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 22:56:15.778513 8382 raft_consensus.cc:504] T a28c08be163445c9884d63b673afd5ad P c988eacb729c44a0bdd072dc42707c58 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c988eacb729c44a0bdd072dc42707c58" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 40825 } } peers { permanent_uuid: "a8594d32d7724c3b831d36b48c70ba60" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 46709 } } peers { permanent_uuid: "0022f56469854dd5b9783b4e88420ef5" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 33009 } }
I20260809 22:56:15.780282 8382 leader_election.cc:288] T a28c08be163445c9884d63b673afd5ad P c988eacb729c44a0bdd072dc42707c58 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a8594d32d7724c3b831d36b48c70ba60 (127.15.113.131:46709), 0022f56469854dd5b9783b4e88420ef5 (127.15.113.130:33009)
I20260809 22:56:15.787322 8362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:56:15.780845 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.113.129:43829 (local address 127.15.113.131:46709)
0809 22:56:15.781076 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0809 22:56:15.781083 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:56:15.781852 (+ 769us) server_negotiation.cc:407] Connection header received
0809 22:56:15.782044 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:56:15.782049 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:56:15.782127 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:56:15.782214 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:56:15.782974 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:56:15.783728 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:56:15.785587 (+ 1859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:56:15.786414 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:56:15.786554 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:56:15.786625 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0809 22:56:15.786956 (+ 331us) server_negotiation.cc:299] Negotiation successful
0809 22:56:15.787080 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94}
I20260809 22:56:15.787523 8385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:56:15.780472 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.131:46709 (local address 127.15.113.129:43829)
0809 22:56:15.781722 (+ 1250us) negotiation.cc:107] Waiting for socket to connect
0809 22:56:15.781744 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0809 22:56:15.781905 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:56:15.782308 (+ 403us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:56:15.782314 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:56:15.782352 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:56:15.782813 (+ 461us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:56:15.782831 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:56:15.783861 (+ 1030us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:56:15.783867 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:56:15.785439 (+ 1572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:56:15.785452 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:56:15.786604 (+ 1152us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:56:15.786610 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:56:15.786780 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:56:15.786816 (+ 36us) client_negotiation.cc:769] Sending connection context
0809 22:56:15.786909 (+ 93us) client_negotiation.cc:240] Negotiation successful
0809 22:56:15.787097 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1093,"mutex_wait_us":3,"thread_start_us":878,"threads_started":1}
I20260809 22:56:15.787541 8371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:56:15.780911 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.113.129:52379 (local address 127.15.113.130:33009)
0809 22:56:15.781139 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0809 22:56:15.781144 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:56:15.782015 (+ 871us) server_negotiation.cc:407] Connection header received
0809 22:56:15.782155 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:56:15.782158 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:56:15.782248 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:56:15.782340 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:56:15.783025 (+ 685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:56:15.783759 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:56:15.785673 (+ 1914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:56:15.786623 (+ 950us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:56:15.786764 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:56:15.786824 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0809 22:56:15.787186 (+ 362us) server_negotiation.cc:299] Negotiation successful
0809 22:56:15.787316 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260809 22:56:15.788146 8386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:56:15.780763 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.130:33009 (local address 127.15.113.129:52379)
0809 22:56:15.781896 (+ 1133us) negotiation.cc:107] Waiting for socket to connect
0809 22:56:15.781912 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 22:56:15.782037 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:56:15.782362 (+ 325us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:56:15.782367 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:56:15.782392 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:56:15.782837 (+ 445us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:56:15.782846 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:56:15.783898 (+ 1052us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:56:15.783904 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:56:15.785501 (+ 1597us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:56:15.785516 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:56:15.786836 (+ 1320us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:56:15.786842 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:56:15.787008 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:56:15.787038 (+ 30us) client_negotiation.cc:769] Sending connection context
0809 22:56:15.787766 (+ 728us) client_negotiation.cc:240] Negotiation successful
0809 22:56:15.787882 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1014,"mutex_wait_us":46,"thread_start_us":838,"threads_started":1}
I20260809 22:56:15.788646 8279 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a28c08be163445c9884d63b673afd5ad" candidate_uuid: "c988eacb729c44a0bdd072dc42707c58" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a8594d32d7724c3b831d36b48c70ba60" is_pre_election: true
I20260809 22:56:15.789225 8279 raft_consensus.cc:2454] T a28c08be163445c9884d63b673afd5ad P a8594d32d7724c3b831d36b48c70ba60 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c988eacb729c44a0bdd072dc42707c58 in term 0.
I20260809 22:56:15.789258 8087 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a28c08be163445c9884d63b673afd5ad" candidate_uuid: "c988eacb729c44a0bdd072dc42707c58" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0022f56469854dd5b9783b4e88420ef5" is_pre_election: true
I20260809 22:56:15.789774 8087 raft_consensus.cc:2454] T a28c08be163445c9884d63b673afd5ad P 0022f56469854dd5b9783b4e88420ef5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c988eacb729c44a0bdd072dc42707c58 in term 0.
I20260809 22:56:15.789834 7803 leader_election.cc:302] T a28c08be163445c9884d63b673afd5ad P c988eacb729c44a0bdd072dc42707c58 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a8594d32d7724c3b831d36b48c70ba60, c988eacb729c44a0bdd072dc42707c58; no voters:
I20260809 22:56:15.790210 8382 raft_consensus.cc:2781] T a28c08be163445c9884d63b673afd5ad P c988eacb729c44a0bdd072dc42707c58 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 22:56:15.790329 8382 raft_consensus.cc:482] T a28c08be163445c9884d63b673afd5ad P c988eacb729c44a0bdd072dc42707c58 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 22:56:15.790432 8382 raft_consensus.cc:3037] T a28c08be163445c9884d63b673afd5ad P c988eacb729c44a0bdd072dc42707c58 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:56:15.792636 8382 raft_consensus.cc:504] T a28c08be163445c9884d63b673afd5ad P c988eacb729c44a0bdd072dc42707c58 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c988eacb729c44a0bdd072dc42707c58" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 40825 } } peers { permanent_uuid: "a8594d32d7724c3b831d36b48c70ba60" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 46709 } } peers { permanent_uuid: "0022f56469854dd5b9783b4e88420ef5" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 33009 } }
I20260809 22:56:15.793247 8382 leader_election.cc:288] T a28c08be163445c9884d63b673afd5ad P c988eacb729c44a0bdd072dc42707c58 [CANDIDATE]: Term 1 election: Requested vote from peers a8594d32d7724c3b831d36b48c70ba60 (127.15.113.131:46709), 0022f56469854dd5b9783b4e88420ef5 (127.15.113.130:33009)
I20260809 22:56:15.793658 8279 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a28c08be163445c9884d63b673afd5ad" candidate_uuid: "c988eacb729c44a0bdd072dc42707c58" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a8594d32d7724c3b831d36b48c70ba60"
I20260809 22:56:15.793732 8087 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a28c08be163445c9884d63b673afd5ad" candidate_uuid: "c988eacb729c44a0bdd072dc42707c58" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0022f56469854dd5b9783b4e88420ef5"
I20260809 22:56:15.793833 8279 raft_consensus.cc:3037] T a28c08be163445c9884d63b673afd5ad P a8594d32d7724c3b831d36b48c70ba60 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:56:15.793929 8087 raft_consensus.cc:3037] T a28c08be163445c9884d63b673afd5ad P 0022f56469854dd5b9783b4e88420ef5 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:56:15.795914 8279 raft_consensus.cc:2454] T a28c08be163445c9884d63b673afd5ad P a8594d32d7724c3b831d36b48c70ba60 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c988eacb729c44a0bdd072dc42707c58 in term 1.
I20260809 22:56:15.796074 8087 raft_consensus.cc:2454] T a28c08be163445c9884d63b673afd5ad P 0022f56469854dd5b9783b4e88420ef5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c988eacb729c44a0bdd072dc42707c58 in term 1.
I20260809 22:56:15.796241 7803 leader_election.cc:302] T a28c08be163445c9884d63b673afd5ad P c988eacb729c44a0bdd072dc42707c58 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a8594d32d7724c3b831d36b48c70ba60, c988eacb729c44a0bdd072dc42707c58; no voters:
I20260809 22:56:15.796507 8382 raft_consensus.cc:2781] T a28c08be163445c9884d63b673afd5ad P c988eacb729c44a0bdd072dc42707c58 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 22:56:15.796962 8382 raft_consensus.cc:686] T a28c08be163445c9884d63b673afd5ad P c988eacb729c44a0bdd072dc42707c58 [term 1 LEADER]: Becoming Leader. State: Replica: c988eacb729c44a0bdd072dc42707c58, State: Running, Role: LEADER
I20260809 22:56:15.797394 8382 consensus_queue.cc:237] T a28c08be163445c9884d63b673afd5ad P c988eacb729c44a0bdd072dc42707c58 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c988eacb729c44a0bdd072dc42707c58" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 40825 } } peers { permanent_uuid: "a8594d32d7724c3b831d36b48c70ba60" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 46709 } } peers { permanent_uuid: "0022f56469854dd5b9783b4e88420ef5" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 33009 } }
I20260809 22:56:15.802050 7692 catalog_manager.cc:5495] T a28c08be163445c9884d63b673afd5ad P c988eacb729c44a0bdd072dc42707c58 reported cstate change: term changed from 0 to 1, leader changed from <none> to c988eacb729c44a0bdd072dc42707c58 (127.15.113.129). New cstate: current_term: 1 leader_uuid: "c988eacb729c44a0bdd072dc42707c58" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c988eacb729c44a0bdd072dc42707c58" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 40825 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a8594d32d7724c3b831d36b48c70ba60" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 46709 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0022f56469854dd5b9783b4e88420ef5" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 33009 } health_report { overall_health: UNKNOWN } } }
I20260809 22:56:15.831818 8360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:56:15.826140 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36500 (local address 127.15.113.129:40825)
0809 22:56:15.826348 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0809 22:56:15.826352 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 22:56:15.826370 (+ 18us) server_negotiation.cc:407] Connection header received
0809 22:56:15.826503 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:56:15.826507 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:56:15.826595 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:56:15.826715 (+ 120us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 22:56:15.827002 (+ 287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:56:15.827696 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:56:15.829798 (+ 2102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:56:15.830213 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:56:15.830304 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:56:15.830806 (+ 502us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 22:56:15.831307 (+ 501us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 22:56:15.831408 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0809 22:56:15.831530 (+ 122us) server_negotiation.cc:299] Negotiation successful
0809 22:56:15.831593 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260809 22:56:15.840597 8087 raft_consensus.cc:1261] T a28c08be163445c9884d63b673afd5ad P 0022f56469854dd5b9783b4e88420ef5 [term 1 FOLLOWER]: Refusing update from remote peer c988eacb729c44a0bdd072dc42707c58: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 22:56:15.840597 8279 raft_consensus.cc:1261] T a28c08be163445c9884d63b673afd5ad P a8594d32d7724c3b831d36b48c70ba60 [term 1 FOLLOWER]: Refusing update from remote peer c988eacb729c44a0bdd072dc42707c58: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 22:56:15.841681 8387 consensus_queue.cc:1035] T a28c08be163445c9884d63b673afd5ad P c988eacb729c44a0bdd072dc42707c58 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a8594d32d7724c3b831d36b48c70ba60" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 46709 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 22:56:15.841989 8382 consensus_queue.cc:1035] T a28c08be163445c9884d63b673afd5ad P c988eacb729c44a0bdd072dc42707c58 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0022f56469854dd5b9783b4e88420ef5" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 33009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 22:56:15.852628 8391 mvcc.cc:205] Tried to move back new op lower bound from 7316751056233164800 to 7316751056069824512. Current Snapshot: MvccSnapshot[applied={T|T < 7316751056233164800}]
I20260809 22:56:15.856215 8393 mvcc.cc:205] Tried to move back new op lower bound from 7316751056233164800 to 7316751056069824512. Current Snapshot: MvccSnapshot[applied={T|T < 7316751056233164800}]
I20260809 22:56:15.866114 7692 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57736:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 22:56:15.866350 7692 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57736:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 22:56:15.869704 7692 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0f3abae906c2437aaf43e0bef2ad018c: Sending DeleteTablet for 3 replicas of tablet a28c08be163445c9884d63b673afd5ad
I20260809 22:56:15.870743 7865 tablet_service.cc:1514] Processing DeleteTablet for tablet a28c08be163445c9884d63b673afd5ad with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:56:15 UTC) from {username='kudu', principal='kudu/127.15.113.190@KRBTEST.COM'} at 127.0.0.1:36496
I20260809 22:56:15.870792 8060 tablet_service.cc:1514] Processing DeleteTablet for tablet a28c08be163445c9884d63b673afd5ad with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:56:15 UTC) from {username='kudu', principal='kudu/127.15.113.190@KRBTEST.COM'} at 127.0.0.1:39538
I20260809 22:56:15.870779 8251 tablet_service.cc:1514] Processing DeleteTablet for tablet a28c08be163445c9884d63b673afd5ad with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:56:15 UTC) from {username='kudu', principal='kudu/127.15.113.190@KRBTEST.COM'} at 127.0.0.1:55368
I20260809 22:56:15.873976 15814 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskItYPJR/build/asan/bin/kudu pid 7775
I20260809 22:56:15.887259 8405 tablet_replica.cc:330] T a28c08be163445c9884d63b673afd5ad P 0022f56469854dd5b9783b4e88420ef5: stopping tablet replica
I20260809 22:56:15.887775 8406 tablet_replica.cc:330] T a28c08be163445c9884d63b673afd5ad P c988eacb729c44a0bdd072dc42707c58: stopping tablet replica
I20260809 22:56:15.888147 8406 raft_consensus.cc:2229] T a28c08be163445c9884d63b673afd5ad P c988eacb729c44a0bdd072dc42707c58 [term 1 LEADER]: Raft consensus shutting down.
I20260809 22:56:15.888695 8406 raft_consensus.cc:2258] T a28c08be163445c9884d63b673afd5ad P c988eacb729c44a0bdd072dc42707c58 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 22:56:15.889313 8405 raft_consensus.cc:2229] T a28c08be163445c9884d63b673afd5ad P 0022f56469854dd5b9783b4e88420ef5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 22:56:15.889917 8405 raft_consensus.cc:2258] T a28c08be163445c9884d63b673afd5ad P 0022f56469854dd5b9783b4e88420ef5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 22:56:15.890964 8407 tablet_replica.cc:330] T a28c08be163445c9884d63b673afd5ad P a8594d32d7724c3b831d36b48c70ba60: stopping tablet replica
I20260809 22:56:15.891372 8407 raft_consensus.cc:2229] T a28c08be163445c9884d63b673afd5ad P a8594d32d7724c3b831d36b48c70ba60 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 22:56:15.891865 8407 raft_consensus.cc:2258] T a28c08be163445c9884d63b673afd5ad P a8594d32d7724c3b831d36b48c70ba60 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 22:56:15.892127 8405 ts_tablet_manager.cc:1905] T a28c08be163445c9884d63b673afd5ad P 0022f56469854dd5b9783b4e88420ef5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 22:56:15.893937 8407 ts_tablet_manager.cc:1905] T a28c08be163445c9884d63b673afd5ad P a8594d32d7724c3b831d36b48c70ba60: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 22:56:15.895866 8406 ts_tablet_manager.cc:1905] T a28c08be163445c9884d63b673afd5ad P c988eacb729c44a0bdd072dc42707c58: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 22:56:15.899551 8405 ts_tablet_manager.cc:1918] T a28c08be163445c9884d63b673afd5ad P 0022f56469854dd5b9783b4e88420ef5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 22:56:15.899758 8405 log.cc:1196] T a28c08be163445c9884d63b673afd5ad P 0022f56469854dd5b9783b4e88420ef5: Deleting WAL directory at /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315935617919-15814-0/minicluster-data/ts-1/wal/wals/a28c08be163445c9884d63b673afd5ad
I20260809 22:56:15.900367 8405 ts_tablet_manager.cc:1939] T a28c08be163445c9884d63b673afd5ad P 0022f56469854dd5b9783b4e88420ef5: Deleting consensus metadata
I20260809 22:56:15.901117 8407 ts_tablet_manager.cc:1918] T a28c08be163445c9884d63b673afd5ad P a8594d32d7724c3b831d36b48c70ba60: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 22:56:15.901346 8407 log.cc:1196] T a28c08be163445c9884d63b673afd5ad P a8594d32d7724c3b831d36b48c70ba60: Deleting WAL directory at /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315935617919-15814-0/minicluster-data/ts-2/wal/wals/a28c08be163445c9884d63b673afd5ad
I20260809 22:56:15.901886 8407 ts_tablet_manager.cc:1939] T a28c08be163445c9884d63b673afd5ad P a8594d32d7724c3b831d36b48c70ba60: Deleting consensus metadata
Aug 09 22:56:15 dist-test-slave-ncp9 krb5kdc[7625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316173, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.113.129@KRBTEST.COM
I20260809 22:56:15.902506 7672 catalog_manager.cc:4841] TS 0022f56469854dd5b9783b4e88420ef5 (127.15.113.130:33009): tablet a28c08be163445c9884d63b673afd5ad (table test-table [id=ab2f6751ac9f45c5b9054304e19affd2]) successfully deleted
I20260809 22:56:15.904637 7675 catalog_manager.cc:4841] TS a8594d32d7724c3b831d36b48c70ba60 (127.15.113.131:46709): tablet a28c08be163445c9884d63b673afd5ad (table test-table [id=ab2f6751ac9f45c5b9054304e19affd2]) successfully deleted
I20260809 22:56:15.907173 8406 ts_tablet_manager.cc:1918] T a28c08be163445c9884d63b673afd5ad P c988eacb729c44a0bdd072dc42707c58: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 22:56:15.907331 8406 log.cc:1196] T a28c08be163445c9884d63b673afd5ad P c988eacb729c44a0bdd072dc42707c58: Deleting WAL directory at /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315935617919-15814-0/minicluster-data/ts-0/wal/wals/a28c08be163445c9884d63b673afd5ad
I20260809 22:56:15.907766 8406 ts_tablet_manager.cc:1939] T a28c08be163445c9884d63b673afd5ad P c988eacb729c44a0bdd072dc42707c58: Deleting consensus metadata
I20260809 22:56:15.909373 7674 catalog_manager.cc:4841] TS c988eacb729c44a0bdd072dc42707c58 (127.15.113.129:40825): tablet a28c08be163445c9884d63b673afd5ad (table test-table [id=ab2f6751ac9f45c5b9054304e19affd2]) successfully deleted
I20260809 22:56:15.911356 8360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:56:15.874760 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36510 (local address 127.15.113.129:40825)
0809 22:56:15.874925 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0809 22:56:15.874931 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:56:15.882785 (+ 7854us) server_negotiation.cc:407] Connection header received
0809 22:56:15.883584 (+ 799us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:56:15.883591 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:56:15.883675 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:56:15.883746 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:56:15.890574 (+ 6828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:56:15.891693 (+ 1119us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:56:15.893871 (+ 2178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:56:15.894524 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:56:15.894625 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:56:15.903679 (+ 9054us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:56:15.903693 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:56:15.903711 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:56:15.903768 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:56:15.908161 (+ 4393us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:56:15.909117 (+ 956us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:56:15.909132 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:56:15.909135 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:56:15.909348 (+ 213us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:56:15.909815 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:56:15.909819 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:56:15.909821 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:56:15.910205 (+ 384us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:56:15.910439 (+ 234us) server_negotiation.cc:1035] Waiting for connection context
0809 22:56:15.910873 (+ 434us) server_negotiation.cc:299] Negotiation successful
0809 22:56:15.911056 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260809 22:56:15.912022 7949 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:56:16.009744 15814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskItYPJR/build/asan/bin/kudu with pid 7775
I20260809 22:56:16.038280 15814 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskItYPJR/build/asan/bin/kudu pid 7963
Aug 09 22:56:16 dist-test-slave-ncp9 krb5kdc[7625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316173, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.113.130@KRBTEST.COM
I20260809 22:56:16.052317 8371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:56:16.039023 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39542 (local address 127.15.113.130:33009)
0809 22:56:16.039212 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0809 22:56:16.039216 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 22:56:16.039268 (+ 52us) server_negotiation.cc:407] Connection header received
0809 22:56:16.039430 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:56:16.039434 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:56:16.039498 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:56:16.039582 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:56:16.040877 (+ 1295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:56:16.041652 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:56:16.042873 (+ 1221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:56:16.043261 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:56:16.043353 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:56:16.047383 (+ 4030us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:56:16.047397 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:56:16.047413 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:56:16.047449 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:56:16.050122 (+ 2673us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:56:16.050739 (+ 617us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:56:16.050746 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:56:16.050748 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:56:16.050858 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:56:16.051120 (+ 262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:56:16.051122 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:56:16.051124 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:56:16.051463 (+ 339us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:56:16.051701 (+ 238us) server_negotiation.cc:1035] Waiting for connection context
0809 22:56:16.051908 (+ 207us) server_negotiation.cc:299] Negotiation successful
0809 22:56:16.052067 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260809 22:56:16.052986 8141 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:56:16.139770 15814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskItYPJR/build/asan/bin/kudu with pid 7963
I20260809 22:56:16.163965 15814 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskItYPJR/build/asan/bin/kudu pid 8159
Aug 09 22:56:16 dist-test-slave-ncp9 krb5kdc[7625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316173, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.113.131@KRBTEST.COM
I20260809 22:56:16.178512 8362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:56:16.164660 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55382 (local address 127.15.113.131:46709)
0809 22:56:16.164817 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0809 22:56:16.164822 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:56:16.164841 (+ 19us) server_negotiation.cc:407] Connection header received
0809 22:56:16.165031 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:56:16.165036 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:56:16.165104 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:56:16.165194 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:56:16.166448 (+ 1254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:56:16.167154 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:56:16.168555 (+ 1401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:56:16.169005 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:56:16.169116 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:56:16.173311 (+ 4195us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:56:16.173327 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:56:16.173345 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:56:16.173378 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:56:16.176030 (+ 2652us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:56:16.176670 (+ 640us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:56:16.176678 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:56:16.176680 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:56:16.176837 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:56:16.177196 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:56:16.177199 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:56:16.177201 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:56:16.177585 (+ 384us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:56:16.177834 (+ 249us) server_negotiation.cc:1035] Waiting for connection context
0809 22:56:16.178091 (+ 257us) server_negotiation.cc:299] Negotiation successful
0809 22:56:16.178263 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260809 22:56:16.179123 8337 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:56:16.266547 15814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskItYPJR/build/asan/bin/kudu with pid 8159
I20260809 22:56:16.288529 15814 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskItYPJR/build/asan/bin/kudu pid 7641
I20260809 22:56:16.289646 7732 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:56:16.397704 15814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskItYPJR/build/asan/bin/kudu with pid 7641
I20260809 22:56:36.423959 15814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskItYPJR/build/asan/bin/kudu
/tmp/dist-test-taskItYPJR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315935617919-15814-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315935617919-15814-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315935617919-15814-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315935617919-15814-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.113.190:39625
--webserver_interface=127.15.113.190
--webserver_port=36581
--keytab_file=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.190.keytab
--principal=kudu/127.15.113.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.113.148:46195
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.113.190:39625
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315935617919-15814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315935617919-15814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315935617919-15814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:56:36.772058 8414 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:56:36.772326 8414 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:56:36.772462 8414 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:56:36.779816 8414 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260809 22:56:36.779917 8414 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260809 22:56:36.779982 8414 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260809 22:56:36.780053 8414 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 22:56:36.780133 8414 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:56:36.780227 8414 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:56:36.780287 8414 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 22:56:36.780361 8414 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 22:56:36.790153 8414 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.113.148:46195
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315935617919-15814-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315935617919-15814-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.15.113.190:39625
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.190.keytab
--principal=kudu/127.15.113.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.113.190:39625
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315935617919-15814-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.113.190
--webserver_port=36581
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315935617919-15814-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:56:36.791337 8414 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:56:36.792644 8414 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:56:36.816881 8424 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:56:36.816944 8426 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:56:36.816835 8423 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:56:36.817728 8414 server_base.cc:1034] running on GCE node
I20260809 22:56:36.818586 8414 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:56:36.820104 8414 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:56:36.821326 8414 hybrid_clock.cc:648] HybridClock initialized: now 1786316196821257 us; error 42 us; skew 500 ppm
I20260809 22:56:36.821697 8414 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:56:36 dist-test-slave-ncp9 krb5kdc[7625](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316196, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.113.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:56:36.825202 8414 init.cc:377] Logged in from keytab as kudu/127.15.113.190@KRBTEST.COM (short username kudu)
I20260809 22:56:36.826673 8414 webserver.cc:460] Webserver started at http://127.15.113.190:36581/ using document root <none> and password file <none>
I20260809 22:56:36.827380 8414 fs_manager.cc:359] Metadata directory not provided
I20260809 22:56:36.827476 8414 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:56:36.833261 8414 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.001s
I20260809 22:56:36.835992 8434 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:56:36.837354 8414 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 22:56:36.837507 8414 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315935617919-15814-0/minicluster-data/master-0/data,/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315935617919-15814-0/minicluster-data/master-0/wal
uuid: "0f3abae906c2437aaf43e0bef2ad018c"
format_stamp: "Formatted at 2026-08-09 22:56:13 on dist-test-slave-ncp9"
server_key: "38c512142111980ce2d84baa62069423"
server_key_iv: "bd6328eedf7756daae78341d56af6aff"
server_key_version: "encryptionkey@0"
I20260809 22:56:36.838506 8414 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315935617919-15814-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315935617919-15814-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315935617919-15814-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:56:36.862275 8414 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:56:36.866050 8414 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:56:36.866312 8414 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:56:36.884044 8414 rpc_server.cc:307] RPC server started. Bound to: 127.15.113.190:39625
I20260809 22:56:36.884068 8517 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.113.190:39625 every 8 connection(s)
I20260809 22:56:36.885556 8414 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315935617919-15814-0/minicluster-data/master-0/data/info.pb
I20260809 22:56:36.892422 8518 sys_catalog.cc:263] Verifying existing consensus state
I20260809 22:56:36.895633 8518 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0f3abae906c2437aaf43e0bef2ad018c: Bootstrap starting.
I20260809 22:56:36.895864 15814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskItYPJR/build/asan/bin/kudu as pid 8414
I20260809 22:56:36.896658 15814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskItYPJR/build/asan/bin/kudu
/tmp/dist-test-taskItYPJR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315935617919-15814-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315935617919-15814-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315935617919-15814-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315935617919-15814-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.113.129:40825
--local_ip_for_outbound_sockets=127.15.113.129
--tserver_master_addrs=127.15.113.190:39625
--webserver_port=34871
--webserver_interface=127.15.113.129
--keytab_file=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.129.keytab
--principal=kudu/127.15.113.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.113.148:46195
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315935617919-15814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315935617919-15814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315935617919-15814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 22:56:36.915352 8518 log.cc:824] T 00000000000000000000000000000000 P 0f3abae906c2437aaf43e0bef2ad018c: Log is configured to *not* fsync() on all Append() calls
I20260809 22:56:36.927683 8518 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0f3abae906c2437aaf43e0bef2ad018c: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 22:56:36.928193 8518 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0f3abae906c2437aaf43e0bef2ad018c: Bootstrap complete.
I20260809 22:56:36.937290 8518 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0f3abae906c2437aaf43e0bef2ad018c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f3abae906c2437aaf43e0bef2ad018c" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 39625 } }
I20260809 22:56:36.938493 8518 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0f3abae906c2437aaf43e0bef2ad018c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0f3abae906c2437aaf43e0bef2ad018c, State: Initialized, Role: FOLLOWER
I20260809 22:56:36.939235 8518 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0f3abae906c2437aaf43e0bef2ad018c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f3abae906c2437aaf43e0bef2ad018c" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 39625 } }
I20260809 22:56:36.939543 8518 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0f3abae906c2437aaf43e0bef2ad018c [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 22:56:36.939721 8518 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0f3abae906c2437aaf43e0bef2ad018c [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 22:56:36.939929 8518 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0f3abae906c2437aaf43e0bef2ad018c [term 1 FOLLOWER]: Advancing to term 2
I20260809 22:56:36.945042 8518 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0f3abae906c2437aaf43e0bef2ad018c [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f3abae906c2437aaf43e0bef2ad018c" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 39625 } }
I20260809 22:56:36.945580 8518 leader_election.cc:302] T 00000000000000000000000000000000 P 0f3abae906c2437aaf43e0bef2ad018c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0f3abae906c2437aaf43e0bef2ad018c; no voters:
I20260809 22:56:36.946161 8518 leader_election.cc:288] T 00000000000000000000000000000000 P 0f3abae906c2437aaf43e0bef2ad018c [CANDIDATE]: Term 2 election: Requested vote from peers
I20260809 22:56:36.946316 8522 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0f3abae906c2437aaf43e0bef2ad018c [term 2 FOLLOWER]: Leader election won for term 2
I20260809 22:56:36.947381 8522 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0f3abae906c2437aaf43e0bef2ad018c [term 2 LEADER]: Becoming Leader. State: Replica: 0f3abae906c2437aaf43e0bef2ad018c, State: Running, Role: LEADER
I20260809 22:56:36.947999 8522 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0f3abae906c2437aaf43e0bef2ad018c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f3abae906c2437aaf43e0bef2ad018c" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 39625 } }
I20260809 22:56:36.948410 8518 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0f3abae906c2437aaf43e0bef2ad018c [sys.catalog]: configured and running, proceeding with master startup.
I20260809 22:56:36.957288 8524 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0f3abae906c2437aaf43e0bef2ad018c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0f3abae906c2437aaf43e0bef2ad018c. Latest consensus state: current_term: 2 leader_uuid: "0f3abae906c2437aaf43e0bef2ad018c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f3abae906c2437aaf43e0bef2ad018c" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 39625 } } }
I20260809 22:56:36.957672 8524 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0f3abae906c2437aaf43e0bef2ad018c [sys.catalog]: This master's current role is: LEADER
I20260809 22:56:36.958127 8523 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0f3abae906c2437aaf43e0bef2ad018c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "0f3abae906c2437aaf43e0bef2ad018c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f3abae906c2437aaf43e0bef2ad018c" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 39625 } } }
I20260809 22:56:36.958447 8523 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0f3abae906c2437aaf43e0bef2ad018c [sys.catalog]: This master's current role is: LEADER
I20260809 22:56:36.974615 8532 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 22:56:36.983139 8532 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 22:56:36.984586 8532 catalog_manager.cc:1259] Loaded cluster ID: 0589e66c0b8b476bb039a06f3f16b159
I20260809 22:56:36.984736 8532 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 22:56:36.988680 8532 catalog_manager.cc:1504] Loading token signing keys...
I20260809 22:56:36.990243 8532 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 0f3abae906c2437aaf43e0bef2ad018c: Loaded TSK: 0
I20260809 22:56:37.020112 8532 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0f3abae906c2437aaf43e0bef2ad018c: Generated new TSK 1
I20260809 22:56:37.020496 8532 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260809 22:56:37.256079 8520 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:56:37.256347 8520 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:56:37.256462 8520 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:56:37.263566 8520 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:56:37.263669 8520 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:56:37.263813 8520 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.113.129
I20260809 22:56:37.273222 8520 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.113.148:46195
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315935617919-15814-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315935617919-15814-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.129.keytab
--principal=kudu/127.15.113.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.113.129:40825
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315935617919-15814-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.113.129
--webserver_port=34871
--tserver_master_addrs=127.15.113.190:39625
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.113.129
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315935617919-15814-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:56:37.274427 8520 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:56:37.275697 8520 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:56:37.288908 8558 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:56:37.289260 8560 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:56:37.289573 8557 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:56:37.290648 8520 server_base.cc:1034] running on GCE node
I20260809 22:56:37.291193 8520 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:56:37.292094 8520 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:56:37.293298 8520 hybrid_clock.cc:648] HybridClock initialized: now 1786316197293229 us; error 58 us; skew 500 ppm
I20260809 22:56:37.293606 8520 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:56:37 dist-test-slave-ncp9 krb5kdc[7625](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316197, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.113.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:56:37.296866 8520 init.cc:377] Logged in from keytab as kudu/127.15.113.129@KRBTEST.COM (short username kudu)
I20260809 22:56:37.298712 8520 webserver.cc:460] Webserver started at http://127.15.113.129:34871/ using document root <none> and password file <none>
I20260809 22:56:37.299583 8520 fs_manager.cc:359] Metadata directory not provided
I20260809 22:56:37.299754 8520 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:56:37.306772 8520 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.004s sys 0.000s
I20260809 22:56:37.310087 8572 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:56:37.311666 8520 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 22:56:37.311846 8520 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315935617919-15814-0/minicluster-data/ts-0/data,/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315935617919-15814-0/minicluster-data/ts-0/wal
uuid: "c988eacb729c44a0bdd072dc42707c58"
format_stamp: "Formatted at 2026-08-09 22:56:14 on dist-test-slave-ncp9"
server_key: "6791de00355b98f99ffb6e181829269d"
server_key_iv: "4880d044c42044fe3964d38b4a81c2dc"
server_key_version: "encryptionkey@0"
I20260809 22:56:37.313249 8520 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315935617919-15814-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315935617919-15814-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315935617919-15814-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:56:37.331446 8520 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:56:37.336570 8520 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:56:37.336953 8520 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:56:37.338189 8520 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:56:37.339880 8520 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:56:37.340004 8520 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:56:37.340116 8520 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:56:37.340195 8520 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:56:37.372417 8520 rpc_server.cc:307] RPC server started. Bound to: 127.15.113.129:40825
I20260809 22:56:37.372473 8721 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.113.129:40825 every 8 connection(s)
I20260809 22:56:37.373926 8520 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315935617919-15814-0/minicluster-data/ts-0/data/info.pb
I20260809 22:56:37.375735 15814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskItYPJR/build/asan/bin/kudu as pid 8520
I20260809 22:56:37.376543 15814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskItYPJR/build/asan/bin/kudu
/tmp/dist-test-taskItYPJR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315935617919-15814-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315935617919-15814-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315935617919-15814-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315935617919-15814-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.113.130:33009
--local_ip_for_outbound_sockets=127.15.113.130
--tserver_master_addrs=127.15.113.190:39625
--webserver_port=45625
--webserver_interface=127.15.113.130
--keytab_file=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.130.keytab
--principal=kudu/127.15.113.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.113.148:46195
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315935617919-15814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315935617919-15814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315935617919-15814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 22:56:37 dist-test-slave-ncp9 krb5kdc[7625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316197, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.113.129@KRBTEST.COM for kudu/127.15.113.190@KRBTEST.COM
I20260809 22:56:37.405619 8727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:56:37.376922 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:39625 (local address 127.15.113.129:49793)
0809 22:56:37.381470 (+ 4548us) negotiation.cc:107] Waiting for socket to connect
0809 22:56:37.381516 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0809 22:56:37.386824 (+ 5308us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:56:37.388330 (+ 1506us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:56:37.388340 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:56:37.389251 (+ 911us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:56:37.389832 (+ 581us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:56:37.389852 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:56:37.391573 (+ 1721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:56:37.391579 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:56:37.393587 (+ 2008us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:56:37.393608 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:56:37.394633 (+ 1025us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:56:37.394641 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:56:37.394963 (+ 322us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:56:37.395723 (+ 760us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:56:37.395755 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:56:37.398335 (+ 2580us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:56:37.401544 (+ 3209us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:56:37.401553 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:56:37.401568 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:56:37.401978 (+ 410us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:56:37.402403 (+ 425us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:56:37.402408 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:56:37.402412 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:56:37.402589 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:56:37.403276 (+ 687us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:56:37.403284 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:56:37.403586 (+ 302us) client_negotiation.cc:769] Sending connection context
0809 22:56:37.403819 (+ 233us) client_negotiation.cc:240] Negotiation successful
0809 22:56:37.404341 (+ 522us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4258,"thread_start_us":4048,"threads_started":1}
I20260809 22:56:37.405625 8726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:56:37.376775 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.113.129:49793 (local address 127.15.113.190:39625)
0809 22:56:37.377638 (+ 863us) server_negotiation.cc:206] Beginning negotiation
0809 22:56:37.377667 (+ 29us) server_negotiation.cc:399] Waiting for connection header
0809 22:56:37.381671 (+ 4004us) server_negotiation.cc:407] Connection header received
0809 22:56:37.387411 (+ 5740us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:56:37.387427 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:56:37.387808 (+ 381us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:56:37.388128 (+ 320us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:56:37.390038 (+ 1910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:56:37.391369 (+ 1331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:56:37.393750 (+ 2381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:56:37.394384 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:56:37.394510 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:56:37.398586 (+ 4076us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:56:37.398594 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:56:37.398607 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:56:37.398655 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:56:37.401416 (+ 2761us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:56:37.402111 (+ 695us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:56:37.402118 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:56:37.402123 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:56:37.402294 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:56:37.402707 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:56:37.402710 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:56:37.402712 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:56:37.403135 (+ 423us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:56:37.403318 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0809 22:56:37.404063 (+ 745us) server_negotiation.cc:299] Negotiation successful
0809 22:56:37.404440 (+ 377us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":610,"thread_start_us":436,"threads_started":1}
I20260809 22:56:37.408556 8722 heartbeater.cc:347] Connected to a master server at 127.15.113.190:39625
I20260809 22:56:37.408955 8722 heartbeater.cc:464] Registering TS with master...
I20260809 22:56:37.409904 8722 heartbeater.cc:511] Master 127.15.113.190:39625 requested a full tablet report, sending...
I20260809 22:56:37.412200 8459 ts_manager.cc:194] Registered new tserver with Master: c988eacb729c44a0bdd072dc42707c58 (127.15.113.129:40825)
I20260809 22:56:37.413753 8459 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.113.129@KRBTEST.COM'} at 127.15.113.129:49793
W20260809 22:56:37.727674 8729 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:56:37.727963 8729 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:56:37.728123 8729 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:56:37.735251 8729 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:56:37.735363 8729 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:56:37.735538 8729 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.113.130
I20260809 22:56:37.745289 8729 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.113.148:46195
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315935617919-15814-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315935617919-15814-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.130.keytab
--principal=kudu/127.15.113.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.113.130:33009
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315935617919-15814-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.113.130
--webserver_port=45625
--tserver_master_addrs=127.15.113.190:39625
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.113.130
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315935617919-15814-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:56:37.746505 8729 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:56:37.747822 8729 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:56:37.760742 8736 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:56:37.760825 8740 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:56:37.760802 8738 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:56:37.761955 8729 server_base.cc:1034] running on GCE node
I20260809 22:56:37.762521 8729 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:56:37.763393 8729 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:56:37.764585 8729 hybrid_clock.cc:648] HybridClock initialized: now 1786316197764535 us; error 45 us; skew 500 ppm
I20260809 22:56:37.764864 8729 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:56:37 dist-test-slave-ncp9 krb5kdc[7625](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316197, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.113.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:56:37.767798 8729 init.cc:377] Logged in from keytab as kudu/127.15.113.130@KRBTEST.COM (short username kudu)
I20260809 22:56:37.769095 8729 webserver.cc:460] Webserver started at http://127.15.113.130:45625/ using document root <none> and password file <none>
I20260809 22:56:37.769706 8729 fs_manager.cc:359] Metadata directory not provided
I20260809 22:56:37.769816 8729 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:56:37.774758 8729 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.002s
I20260809 22:56:37.777343 8748 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:56:37.778586 8729 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.001s
I20260809 22:56:37.778728 8729 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315935617919-15814-0/minicluster-data/ts-1/data,/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315935617919-15814-0/minicluster-data/ts-1/wal
uuid: "0022f56469854dd5b9783b4e88420ef5"
format_stamp: "Formatted at 2026-08-09 22:56:14 on dist-test-slave-ncp9"
server_key: "9a840933e2926fdf96995a968373eb05"
server_key_iv: "b62d759303acd0b00a4efd1623e5c932"
server_key_version: "encryptionkey@0"
I20260809 22:56:37.779630 8729 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315935617919-15814-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315935617919-15814-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315935617919-15814-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:56:37.807291 8729 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:56:37.811177 8729 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:56:37.811398 8729 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:56:37.812305 8729 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:56:37.813730 8729 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:56:37.813817 8729 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:56:37.813897 8729 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:56:37.813953 8729 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:56:37.845075 8729 rpc_server.cc:307] RPC server started. Bound to: 127.15.113.130:33009
I20260809 22:56:37.845134 8908 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.113.130:33009 every 8 connection(s)
I20260809 22:56:37.846439 8729 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315935617919-15814-0/minicluster-data/ts-1/data/info.pb
I20260809 22:56:37.847422 15814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskItYPJR/build/asan/bin/kudu as pid 8729
I20260809 22:56:37.848065 15814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskItYPJR/build/asan/bin/kudu
/tmp/dist-test-taskItYPJR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315935617919-15814-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315935617919-15814-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315935617919-15814-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315935617919-15814-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.113.131:46709
--local_ip_for_outbound_sockets=127.15.113.131
--tserver_master_addrs=127.15.113.190:39625
--webserver_port=33209
--webserver_interface=127.15.113.131
--keytab_file=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.131.keytab
--principal=kudu/127.15.113.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.113.148:46195
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315935617919-15814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315935617919-15814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315935617919-15814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 22:56:37 dist-test-slave-ncp9 krb5kdc[7625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316197, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.113.130@KRBTEST.COM for kudu/127.15.113.190@KRBTEST.COM
I20260809 22:56:37.866236 8726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:56:37.849002 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.113.130:53549 (local address 127.15.113.190:39625)
0809 22:56:37.849202 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0809 22:56:37.849209 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:56:37.850509 (+ 1300us) server_negotiation.cc:407] Connection header received
0809 22:56:37.852395 (+ 1886us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:56:37.852401 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:56:37.852483 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:56:37.852598 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:56:37.854118 (+ 1520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:56:37.855057 (+ 939us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:56:37.856727 (+ 1670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:56:37.857306 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:56:37.857428 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:56:37.860863 (+ 3435us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:56:37.860868 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:56:37.860871 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:56:37.860937 (+ 66us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:56:37.863233 (+ 2296us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:56:37.863991 (+ 758us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:56:37.863997 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:56:37.864001 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:56:37.864134 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:56:37.864548 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:56:37.864554 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:56:37.864558 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:56:37.864833 (+ 275us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:56:37.864995 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0809 22:56:37.865666 (+ 671us) server_negotiation.cc:299] Negotiation successful
0809 22:56:37.865897 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260809 22:56:37.867112 8911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:56:37.849514 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:39625 (local address 127.15.113.130:53549)
0809 22:56:37.850335 (+ 821us) negotiation.cc:107] Waiting for socket to connect
0809 22:56:37.850390 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0809 22:56:37.852133 (+ 1743us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:56:37.852800 (+ 667us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:56:37.852808 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:56:37.853388 (+ 580us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:56:37.853956 (+ 568us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:56:37.853973 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:56:37.855255 (+ 1282us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:56:37.855260 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:56:37.856562 (+ 1302us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:56:37.856578 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:56:37.857513 (+ 935us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:56:37.857518 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:56:37.857724 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:56:37.858331 (+ 607us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:56:37.858352 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:56:37.860645 (+ 2293us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:56:37.863373 (+ 2728us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:56:37.863382 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:56:37.863396 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:56:37.863799 (+ 403us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:56:37.864240 (+ 441us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:56:37.864246 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:56:37.864250 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:56:37.864441 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:56:37.864993 (+ 552us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:56:37.865001 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:56:37.865315 (+ 314us) client_negotiation.cc:769] Sending connection context
0809 22:56:37.865548 (+ 233us) client_negotiation.cc:240] Negotiation successful
0809 22:56:37.866024 (+ 476us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":533,"thread_start_us":337,"threads_started":1}
I20260809 22:56:37.868541 8909 heartbeater.cc:347] Connected to a master server at 127.15.113.190:39625
I20260809 22:56:37.868810 8909 heartbeater.cc:464] Registering TS with master...
I20260809 22:56:37.869480 8909 heartbeater.cc:511] Master 127.15.113.190:39625 requested a full tablet report, sending...
I20260809 22:56:37.870919 8459 ts_manager.cc:194] Registered new tserver with Master: 0022f56469854dd5b9783b4e88420ef5 (127.15.113.130:33009)
I20260809 22:56:37.871691 8459 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.113.130@KRBTEST.COM'} at 127.15.113.130:53549
W20260809 22:56:38.188843 8913 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:56:38.189138 8913 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:56:38.189287 8913 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:56:38.196249 8913 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:56:38.196360 8913 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:56:38.196520 8913 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.113.131
I20260809 22:56:38.205875 8913 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.113.148:46195
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315935617919-15814-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315935617919-15814-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.131.keytab
--principal=kudu/127.15.113.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.113.131:46709
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315935617919-15814-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.113.131
--webserver_port=33209
--tserver_master_addrs=127.15.113.190:39625
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.113.131
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315935617919-15814-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:56:38.207089 8913 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:56:38.208369 8913 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:56:38.220857 8918 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:56:38.220944 8920 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:56:38.220995 8924 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:56:38.221992 8913 server_base.cc:1034] running on GCE node
I20260809 22:56:38.222576 8913 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:56:38.223433 8913 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:56:38.224625 8913 hybrid_clock.cc:648] HybridClock initialized: now 1786316198224567 us; error 54 us; skew 500 ppm
I20260809 22:56:38.224941 8913 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:56:38 dist-test-slave-ncp9 krb5kdc[7625](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316198, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.113.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:56:38.227882 8913 init.cc:377] Logged in from keytab as kudu/127.15.113.131@KRBTEST.COM (short username kudu)
I20260809 22:56:38.229246 8913 webserver.cc:460] Webserver started at http://127.15.113.131:33209/ using document root <none> and password file <none>
I20260809 22:56:38.229884 8913 fs_manager.cc:359] Metadata directory not provided
I20260809 22:56:38.230011 8913 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:56:38.235018 8913 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260809 22:56:38.237546 8934 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:56:38.238772 8913 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 22:56:38.238915 8913 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315935617919-15814-0/minicluster-data/ts-2/data,/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315935617919-15814-0/minicluster-data/ts-2/wal
uuid: "a8594d32d7724c3b831d36b48c70ba60"
format_stamp: "Formatted at 2026-08-09 22:56:15 on dist-test-slave-ncp9"
server_key: "a9b5faab3611544f4151bd0358783ef9"
server_key_iv: "a82a94e30a001d1c2576a2c51c70eeee"
server_key_version: "encryptionkey@0"
I20260809 22:56:38.239821 8913 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315935617919-15814-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315935617919-15814-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315935617919-15814-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:56:38.264966 8913 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:56:38.268507 8913 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:56:38.268761 8913 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:56:38.269685 8913 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:56:38.270911 8913 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:56:38.271014 8913 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:56:38.271118 8913 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:56:38.271189 8913 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:56:38.301319 8913 rpc_server.cc:307] RPC server started. Bound to: 127.15.113.131:46709
I20260809 22:56:38.301398 9088 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.113.131:46709 every 8 connection(s)
I20260809 22:56:38.302695 8913 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315935617919-15814-0/minicluster-data/ts-2/data/info.pb
I20260809 22:56:38.307683 15814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskItYPJR/build/asan/bin/kudu as pid 8913
Aug 09 22:56:38 dist-test-slave-ncp9 krb5kdc[7625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316198, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.113.131@KRBTEST.COM for kudu/127.15.113.190@KRBTEST.COM
I20260809 22:56:38.322906 9093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:56:38.308542 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59632 (local address 127.15.113.190:39625)
0809 22:56:38.308954 (+ 412us) server_negotiation.cc:206] Beginning negotiation
0809 22:56:38.308960 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:56:38.309412 (+ 452us) server_negotiation.cc:407] Connection header received
0809 22:56:38.309594 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:56:38.309600 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:56:38.309655 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:56:38.309739 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:56:38.311762 (+ 2023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:56:38.312477 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:56:38.314174 (+ 1697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:56:38.314803 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:56:38.314931 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:56:38.317580 (+ 2649us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:56:38.317585 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:56:38.317588 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:56:38.317627 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:56:38.320279 (+ 2652us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:56:38.320948 (+ 669us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:56:38.320954 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:56:38.320957 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:56:38.321091 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:56:38.321450 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:56:38.321453 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:56:38.321456 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:56:38.321705 (+ 249us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:56:38.321935 (+ 230us) server_negotiation.cc:1035] Waiting for connection context
0809 22:56:38.322281 (+ 346us) server_negotiation.cc:299] Negotiation successful
0809 22:56:38.322480 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":293,"thread_start_us":207,"threads_started":1}
I20260809 22:56:38.323009 8726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:56:38.305045 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.113.131:38623 (local address 127.15.113.190:39625)
0809 22:56:38.305259 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0809 22:56:38.305265 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:56:38.306335 (+ 1070us) server_negotiation.cc:407] Connection header received
0809 22:56:38.308141 (+ 1806us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:56:38.308146 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:56:38.308235 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:56:38.308323 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:56:38.310114 (+ 1791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:56:38.311261 (+ 1147us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:56:38.312740 (+ 1479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:56:38.313412 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:56:38.313544 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:56:38.316941 (+ 3397us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:56:38.316945 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:56:38.316949 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:56:38.316980 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:56:38.320102 (+ 3122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:56:38.320784 (+ 682us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:56:38.320788 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:56:38.320790 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:56:38.320945 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:56:38.321352 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:56:38.321357 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:56:38.321361 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:56:38.321630 (+ 269us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:56:38.321755 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0809 22:56:38.322420 (+ 665us) server_negotiation.cc:299] Negotiation successful
0809 22:56:38.322627 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72,"spinlock_wait_cycles":6016}
I20260809 22:56:38.324126 9092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:56:38.305487 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:39625 (local address 127.15.113.131:38623)
0809 22:56:38.306180 (+ 693us) negotiation.cc:107] Waiting for socket to connect
0809 22:56:38.306219 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0809 22:56:38.307876 (+ 1657us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:56:38.308609 (+ 733us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:56:38.308633 (+ 24us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:56:38.309375 (+ 742us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:56:38.309956 (+ 581us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:56:38.309974 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:56:38.311388 (+ 1414us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:56:38.311391 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:56:38.312608 (+ 1217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:56:38.312631 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:56:38.313607 (+ 976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:56:38.313612 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:56:38.313822 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:56:38.314447 (+ 625us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:56:38.314477 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:56:38.316721 (+ 2244us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:56:38.320249 (+ 3528us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:56:38.320258 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:56:38.320274 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:56:38.320670 (+ 396us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:56:38.321046 (+ 376us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:56:38.321051 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:56:38.321056 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:56:38.321243 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:56:38.321764 (+ 521us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:56:38.321772 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:56:38.322074 (+ 302us) client_negotiation.cc:769] Sending connection context
0809 22:56:38.322323 (+ 249us) client_negotiation.cc:240] Negotiation successful
0809 22:56:38.322824 (+ 501us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":447,"thread_start_us":261,"threads_started":1}
I20260809 22:56:38.325539 9089 heartbeater.cc:347] Connected to a master server at 127.15.113.190:39625
I20260809 22:56:38.325841 9089 heartbeater.cc:464] Registering TS with master...
I20260809 22:56:38.326504 9089 heartbeater.cc:511] Master 127.15.113.190:39625 requested a full tablet report, sending...
I20260809 22:56:38.327852 8459 ts_manager.cc:194] Registered new tserver with Master: a8594d32d7724c3b831d36b48c70ba60 (127.15.113.131:46709)
I20260809 22:56:38.328606 8459 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.113.131@KRBTEST.COM'} at 127.15.113.131:38623
I20260809 22:56:38.337353 15814 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 22:56:38.343381 9093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:56:38.338245 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59636 (local address 127.15.113.190:39625)
0809 22:56:38.338443 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0809 22:56:38.338449 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:56:38.338744 (+ 295us) server_negotiation.cc:407] Connection header received
0809 22:56:38.338921 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:56:38.338924 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:56:38.339044 (+ 120us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:56:38.339157 (+ 113us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 22:56:38.339455 (+ 298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:56:38.340109 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:56:38.341606 (+ 1497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:56:38.342016 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:56:38.342098 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:56:38.342584 (+ 486us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 22:56:38.342862 (+ 278us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0809 22:56:38.343141 (+ 279us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:59636: token expired
Metrics: {"server-negotiator.queue_time_us":57}
W20260809 22:56:38.343509 9093 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:59636: token expired
W20260809 22:56:38.343910 8351 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.15.113.190:39625)
I20260809 22:56:38.355872 9093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:56:38.344457 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59648 (local address 127.15.113.190:39625)
0809 22:56:38.344597 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0809 22:56:38.344603 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:56:38.344619 (+ 16us) server_negotiation.cc:407] Connection header received
0809 22:56:38.344767 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:56:38.344772 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:56:38.344833 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:56:38.344921 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:56:38.346194 (+ 1273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:56:38.346849 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:56:38.348195 (+ 1346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:56:38.348592 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:56:38.348699 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:56:38.351162 (+ 2463us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:56:38.351168 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:56:38.351172 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:56:38.351200 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:56:38.353414 (+ 2214us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:56:38.354084 (+ 670us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:56:38.354089 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:56:38.354093 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:56:38.354217 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:56:38.354576 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:56:38.354581 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:56:38.354584 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:56:38.354869 (+ 285us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:56:38.355103 (+ 234us) server_negotiation.cc:1035] Waiting for connection context
0809 22:56:38.355403 (+ 300us) server_negotiation.cc:299] Negotiation successful
0809 22:56:38.355595 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260809 22:56:38.358901 8459 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59648:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 22:56:38.366106 8459 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 22:56:38.417232 8722 heartbeater.cc:503] Master 127.15.113.190:39625 was elected leader, sending a full tablet report...
I20260809 22:56:38.420274 9103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:56:38.377841 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36106 (local address 127.15.113.129:40825)
0809 22:56:38.409886 (+ 32045us) server_negotiation.cc:206] Beginning negotiation
0809 22:56:38.409896 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 22:56:38.409931 (+ 35us) server_negotiation.cc:407] Connection header received
0809 22:56:38.410042 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:56:38.410051 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:56:38.410357 (+ 306us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:56:38.410562 (+ 205us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:56:38.411830 (+ 1268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:56:38.413577 (+ 1747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:56:38.417148 (+ 3571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:56:38.419116 (+ 1968us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:56:38.419299 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:56:38.419479 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0809 22:56:38.419817 (+ 338us) server_negotiation.cc:299] Negotiation successful
0809 22:56:38.419973 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31852,"thread_start_us":33725,"threads_started":1}
I20260809 22:56:38.420418 9107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:56:38.377725 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.129:40825 (local address 127.0.0.1:36106)
0809 22:56:38.407245 (+ 29520us) negotiation.cc:107] Waiting for socket to connect
0809 22:56:38.407307 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0809 22:56:38.407485 (+ 178us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:56:38.410875 (+ 3390us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:56:38.410884 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:56:38.410919 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:56:38.411397 (+ 478us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:56:38.411410 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:56:38.414434 (+ 3024us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:56:38.414445 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:56:38.416962 (+ 2517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:56:38.416977 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:56:38.419374 (+ 2397us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:56:38.419382 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:56:38.419594 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:56:38.419633 (+ 39us) client_negotiation.cc:769] Sending connection context
0809 22:56:38.419745 (+ 112us) client_negotiation.cc:240] Negotiation successful
0809 22:56:38.419945 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29208,"mutex_wait_us":3,"thread_start_us":38314,"threads_started":1}
I20260809 22:56:38.423636 9104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:56:38.377803 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.130:33009 (local address 127.0.0.1:44080)
0809 22:56:38.414131 (+ 36328us) negotiation.cc:107] Waiting for socket to connect
0809 22:56:38.414149 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 22:56:38.414237 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:56:38.415367 (+ 1130us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:56:38.415372 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:56:38.415396 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:56:38.416060 (+ 664us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:56:38.416072 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:56:38.419144 (+ 3072us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:56:38.419157 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:56:38.421366 (+ 2209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:56:38.421380 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:56:38.422883 (+ 1503us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:56:38.422890 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:56:38.423053 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:56:38.423087 (+ 34us) client_negotiation.cc:769] Sending connection context
0809 22:56:38.423165 (+ 78us) client_negotiation.cc:240] Negotiation successful
0809 22:56:38.423287 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36190,"mutex_wait_us":286,"thread_start_us":36048,"threads_started":1}
I20260809 22:56:38.423671 9105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:56:38.378191 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44080 (local address 127.15.113.130:33009)
0809 22:56:38.414765 (+ 36574us) server_negotiation.cc:206] Beginning negotiation
0809 22:56:38.414774 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 22:56:38.414802 (+ 28us) server_negotiation.cc:407] Connection header received
0809 22:56:38.414895 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:56:38.414903 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:56:38.415148 (+ 245us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:56:38.415673 (+ 525us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:56:38.416563 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:56:38.418754 (+ 2191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:56:38.421542 (+ 2788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:56:38.422675 (+ 1133us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:56:38.422829 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:56:38.422995 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0809 22:56:38.423231 (+ 236us) server_negotiation.cc:299] Negotiation successful
0809 22:56:38.423395 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36400,"thread_start_us":36296,"threads_started":1}
I20260809 22:56:38.424825 9102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:56:38.379524 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.131:46709 (local address 127.0.0.1:44304)
0809 22:56:38.416339 (+ 36815us) negotiation.cc:107] Waiting for socket to connect
0809 22:56:38.416355 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 22:56:38.416554 (+ 199us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:56:38.417519 (+ 965us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:56:38.417523 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:56:38.417548 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:56:38.417942 (+ 394us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:56:38.417955 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:56:38.419944 (+ 1989us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:56:38.419949 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:56:38.422009 (+ 2060us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:56:38.422024 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:56:38.424060 (+ 2036us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:56:38.424066 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:56:38.424236 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:56:38.424268 (+ 32us) client_negotiation.cc:769] Sending connection context
0809 22:56:38.424361 (+ 93us) client_negotiation.cc:240] Negotiation successful
0809 22:56:38.424481 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36693,"mutex_wait_us":1,"thread_start_us":29208,"threads_started":1}
I20260809 22:56:38.424463 8630 tablet_service.cc:1467] Processing CreateTablet for tablet 692fbd11681248d18ddf09fac28423fd (DEFAULT_TABLE table=test-table [id=5d0fdb2ec6c9473da9eb25d71c582a81]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:56:38.425899 8630 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 692fbd11681248d18ddf09fac28423fd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:56:38.426843 8813 tablet_service.cc:1467] Processing CreateTablet for tablet 692fbd11681248d18ddf09fac28423fd (DEFAULT_TABLE table=test-table [id=5d0fdb2ec6c9473da9eb25d71c582a81]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:56:38.427469 9109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:56:38.379733 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44304 (local address 127.15.113.131:46709)
0809 22:56:38.408045 (+ 28312us) server_negotiation.cc:206] Beginning negotiation
0809 22:56:38.408055 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 22:56:38.416454 (+ 8399us) server_negotiation.cc:407] Connection header received
0809 22:56:38.416761 (+ 307us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:56:38.416769 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:56:38.417007 (+ 238us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:56:38.417173 (+ 166us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:56:38.418156 (+ 983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:56:38.419730 (+ 1574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:56:38.422197 (+ 2467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:56:38.423540 (+ 1343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:56:38.423718 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:56:38.423879 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0809 22:56:38.424418 (+ 539us) server_negotiation.cc:299] Negotiation successful
0809 22:56:38.424572 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28107,"thread_start_us":28337,"threads_started":1}
I20260809 22:56:38.428130 8813 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 692fbd11681248d18ddf09fac28423fd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:56:38.430917 9004 tablet_service.cc:1467] Processing CreateTablet for tablet 692fbd11681248d18ddf09fac28423fd (DEFAULT_TABLE table=test-table [id=5d0fdb2ec6c9473da9eb25d71c582a81]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:56:38.432204 9004 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 692fbd11681248d18ddf09fac28423fd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:56:38.437897 9110 tablet_bootstrap.cc:492] T 692fbd11681248d18ddf09fac28423fd P c988eacb729c44a0bdd072dc42707c58: Bootstrap starting.
I20260809 22:56:38.441207 9110 tablet_bootstrap.cc:654] T 692fbd11681248d18ddf09fac28423fd P c988eacb729c44a0bdd072dc42707c58: Neither blocks nor log segments found. Creating new log.
I20260809 22:56:38.442255 9110 log.cc:824] T 692fbd11681248d18ddf09fac28423fd P c988eacb729c44a0bdd072dc42707c58: Log is configured to *not* fsync() on all Append() calls
I20260809 22:56:38.443673 9112 tablet_bootstrap.cc:492] T 692fbd11681248d18ddf09fac28423fd P 0022f56469854dd5b9783b4e88420ef5: Bootstrap starting.
I20260809 22:56:38.445250 9110 tablet_bootstrap.cc:492] T 692fbd11681248d18ddf09fac28423fd P c988eacb729c44a0bdd072dc42707c58: No bootstrap required, opened a new log
I20260809 22:56:38.445478 9110 ts_tablet_manager.cc:1397] T 692fbd11681248d18ddf09fac28423fd P c988eacb729c44a0bdd072dc42707c58: Time spent bootstrapping tablet: real 0.008s user 0.007s sys 0.000s
I20260809 22:56:38.446566 9114 tablet_bootstrap.cc:492] T 692fbd11681248d18ddf09fac28423fd P a8594d32d7724c3b831d36b48c70ba60: Bootstrap starting.
I20260809 22:56:38.448246 9112 tablet_bootstrap.cc:654] T 692fbd11681248d18ddf09fac28423fd P 0022f56469854dd5b9783b4e88420ef5: Neither blocks nor log segments found. Creating new log.
I20260809 22:56:38.449303 9112 log.cc:824] T 692fbd11681248d18ddf09fac28423fd P 0022f56469854dd5b9783b4e88420ef5: Log is configured to *not* fsync() on all Append() calls
I20260809 22:56:38.449954 9114 tablet_bootstrap.cc:654] T 692fbd11681248d18ddf09fac28423fd P a8594d32d7724c3b831d36b48c70ba60: Neither blocks nor log segments found. Creating new log.
I20260809 22:56:38.451094 9114 log.cc:824] T 692fbd11681248d18ddf09fac28423fd P a8594d32d7724c3b831d36b48c70ba60: Log is configured to *not* fsync() on all Append() calls
I20260809 22:56:38.451776 9112 tablet_bootstrap.cc:492] T 692fbd11681248d18ddf09fac28423fd P 0022f56469854dd5b9783b4e88420ef5: No bootstrap required, opened a new log
I20260809 22:56:38.452042 9112 ts_tablet_manager.cc:1397] T 692fbd11681248d18ddf09fac28423fd P 0022f56469854dd5b9783b4e88420ef5: Time spent bootstrapping tablet: real 0.009s user 0.005s sys 0.000s
I20260809 22:56:38.453953 9114 tablet_bootstrap.cc:492] T 692fbd11681248d18ddf09fac28423fd P a8594d32d7724c3b831d36b48c70ba60: No bootstrap required, opened a new log
I20260809 22:56:38.454201 9114 ts_tablet_manager.cc:1397] T 692fbd11681248d18ddf09fac28423fd P a8594d32d7724c3b831d36b48c70ba60: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.001s
I20260809 22:56:38.455278 9110 raft_consensus.cc:348] T 692fbd11681248d18ddf09fac28423fd P c988eacb729c44a0bdd072dc42707c58 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c988eacb729c44a0bdd072dc42707c58" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 40825 } } peers { permanent_uuid: "0022f56469854dd5b9783b4e88420ef5" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 33009 } } peers { permanent_uuid: "a8594d32d7724c3b831d36b48c70ba60" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 46709 } }
I20260809 22:56:38.455700 9110 raft_consensus.cc:374] T 692fbd11681248d18ddf09fac28423fd P c988eacb729c44a0bdd072dc42707c58 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:56:38.455816 9110 raft_consensus.cc:729] T 692fbd11681248d18ddf09fac28423fd P c988eacb729c44a0bdd072dc42707c58 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c988eacb729c44a0bdd072dc42707c58, State: Initialized, Role: FOLLOWER
I20260809 22:56:38.456516 9110 consensus_queue.cc:260] T 692fbd11681248d18ddf09fac28423fd P c988eacb729c44a0bdd072dc42707c58 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c988eacb729c44a0bdd072dc42707c58" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 40825 } } peers { permanent_uuid: "0022f56469854dd5b9783b4e88420ef5" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 33009 } } peers { permanent_uuid: "a8594d32d7724c3b831d36b48c70ba60" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 46709 } }
I20260809 22:56:38.458441 9110 ts_tablet_manager.cc:1428] T 692fbd11681248d18ddf09fac28423fd P c988eacb729c44a0bdd072dc42707c58: Time spent starting tablet: real 0.013s user 0.012s sys 0.000s
I20260809 22:56:38.459915 9112 raft_consensus.cc:348] T 692fbd11681248d18ddf09fac28423fd P 0022f56469854dd5b9783b4e88420ef5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c988eacb729c44a0bdd072dc42707c58" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 40825 } } peers { permanent_uuid: "0022f56469854dd5b9783b4e88420ef5" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 33009 } } peers { permanent_uuid: "a8594d32d7724c3b831d36b48c70ba60" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 46709 } }
I20260809 22:56:38.460216 9112 raft_consensus.cc:374] T 692fbd11681248d18ddf09fac28423fd P 0022f56469854dd5b9783b4e88420ef5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:56:38.460309 9112 raft_consensus.cc:729] T 692fbd11681248d18ddf09fac28423fd P 0022f56469854dd5b9783b4e88420ef5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0022f56469854dd5b9783b4e88420ef5, State: Initialized, Role: FOLLOWER
I20260809 22:56:38.460815 9112 consensus_queue.cc:260] T 692fbd11681248d18ddf09fac28423fd P 0022f56469854dd5b9783b4e88420ef5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c988eacb729c44a0bdd072dc42707c58" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 40825 } } peers { permanent_uuid: "0022f56469854dd5b9783b4e88420ef5" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 33009 } } peers { permanent_uuid: "a8594d32d7724c3b831d36b48c70ba60" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 46709 } }
I20260809 22:56:38.461820 8909 heartbeater.cc:503] Master 127.15.113.190:39625 was elected leader, sending a full tablet report...
I20260809 22:56:38.462452 9112 ts_tablet_manager.cc:1428] T 692fbd11681248d18ddf09fac28423fd P 0022f56469854dd5b9783b4e88420ef5: Time spent starting tablet: real 0.010s user 0.007s sys 0.004s
I20260809 22:56:38.463909 9114 raft_consensus.cc:348] T 692fbd11681248d18ddf09fac28423fd P a8594d32d7724c3b831d36b48c70ba60 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c988eacb729c44a0bdd072dc42707c58" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 40825 } } peers { permanent_uuid: "0022f56469854dd5b9783b4e88420ef5" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 33009 } } peers { permanent_uuid: "a8594d32d7724c3b831d36b48c70ba60" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 46709 } }
I20260809 22:56:38.464212 9114 raft_consensus.cc:374] T 692fbd11681248d18ddf09fac28423fd P a8594d32d7724c3b831d36b48c70ba60 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:56:38.464315 9114 raft_consensus.cc:729] T 692fbd11681248d18ddf09fac28423fd P a8594d32d7724c3b831d36b48c70ba60 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a8594d32d7724c3b831d36b48c70ba60, State: Initialized, Role: FOLLOWER
I20260809 22:56:38.464876 9114 consensus_queue.cc:260] T 692fbd11681248d18ddf09fac28423fd P a8594d32d7724c3b831d36b48c70ba60 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c988eacb729c44a0bdd072dc42707c58" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 40825 } } peers { permanent_uuid: "0022f56469854dd5b9783b4e88420ef5" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 33009 } } peers { permanent_uuid: "a8594d32d7724c3b831d36b48c70ba60" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 46709 } }
I20260809 22:56:38.465778 9089 heartbeater.cc:503] Master 127.15.113.190:39625 was elected leader, sending a full tablet report...
I20260809 22:56:38.466442 9114 ts_tablet_manager.cc:1428] T 692fbd11681248d18ddf09fac28423fd P a8594d32d7724c3b831d36b48c70ba60: Time spent starting tablet: real 0.012s user 0.005s sys 0.007s
W20260809 22:56:38.554224 9090 tablet.cc:2369] T 692fbd11681248d18ddf09fac28423fd P a8594d32d7724c3b831d36b48c70ba60: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 22:56:38.598373 8910 tablet.cc:2369] T 692fbd11681248d18ddf09fac28423fd P 0022f56469854dd5b9783b4e88420ef5: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 22:56:38.614899 9121 raft_consensus.cc:482] T 692fbd11681248d18ddf09fac28423fd P a8594d32d7724c3b831d36b48c70ba60 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 22:56:38.615172 9121 raft_consensus.cc:504] T 692fbd11681248d18ddf09fac28423fd P a8594d32d7724c3b831d36b48c70ba60 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c988eacb729c44a0bdd072dc42707c58" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 40825 } } peers { permanent_uuid: "0022f56469854dd5b9783b4e88420ef5" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 33009 } } peers { permanent_uuid: "a8594d32d7724c3b831d36b48c70ba60" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 46709 } }
I20260809 22:56:38.616925 9121 leader_election.cc:288] T 692fbd11681248d18ddf09fac28423fd P a8594d32d7724c3b831d36b48c70ba60 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c988eacb729c44a0bdd072dc42707c58 (127.15.113.129:40825), 0022f56469854dd5b9783b4e88420ef5 (127.15.113.130:33009)
I20260809 22:56:38.623083 9092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:56:38.617347 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.129:40825 (local address 127.15.113.131:35449)
0809 22:56:38.617520 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0809 22:56:38.617539 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 22:56:38.617666 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:56:38.618106 (+ 440us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:56:38.618112 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:56:38.618146 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:56:38.618337 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:56:38.618345 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:56:38.619557 (+ 1212us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:56:38.619561 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:56:38.621344 (+ 1783us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:56:38.621359 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:56:38.622558 (+ 1199us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:56:38.622562 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:56:38.622693 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:56:38.622718 (+ 25us) client_negotiation.cc:769] Sending connection context
0809 22:56:38.622787 (+ 69us) client_negotiation.cc:240] Negotiation successful
0809 22:56:38.622879 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33,"mutex_wait_us":1}
I20260809 22:56:38.623207 9103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:56:38.617377 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.113.131:35449 (local address 127.15.113.129:40825)
0809 22:56:38.617581 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0809 22:56:38.617586 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:56:38.617607 (+ 21us) server_negotiation.cc:407] Connection header received
0809 22:56:38.617868 (+ 261us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:56:38.617874 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:56:38.617957 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:56:38.618054 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:56:38.618507 (+ 453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:56:38.619437 (+ 930us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:56:38.621511 (+ 2074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:56:38.622423 (+ 912us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:56:38.622575 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:56:38.622646 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0809 22:56:38.622829 (+ 183us) server_negotiation.cc:299] Negotiation successful
0809 22:56:38.622951 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260809 22:56:38.624011 9105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:56:38.617587 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.113.131:43975 (local address 127.15.113.130:33009)
0809 22:56:38.617778 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0809 22:56:38.617784 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:56:38.618410 (+ 626us) server_negotiation.cc:407] Connection header received
0809 22:56:38.618631 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:56:38.618636 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:56:38.618721 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:56:38.618805 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:56:38.619362 (+ 557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:56:38.620105 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:56:38.622119 (+ 2014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:56:38.623232 (+ 1113us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:56:38.623384 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:56:38.623452 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0809 22:56:38.623725 (+ 273us) server_negotiation.cc:299] Negotiation successful
0809 22:56:38.623836 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260809 22:56:38.624151 9122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:56:38.617351 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.130:33009 (local address 127.15.113.131:43975)
0809 22:56:38.618296 (+ 945us) negotiation.cc:107] Waiting for socket to connect
0809 22:56:38.618314 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 22:56:38.618447 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:56:38.618834 (+ 387us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:56:38.618837 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:56:38.618858 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:56:38.619212 (+ 354us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:56:38.619222 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:56:38.620240 (+ 1018us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:56:38.620246 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:56:38.621969 (+ 1723us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:56:38.621981 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:56:38.623458 (+ 1477us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:56:38.623462 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:56:38.623575 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:56:38.623598 (+ 23us) client_negotiation.cc:769] Sending connection context
0809 22:56:38.623672 (+ 74us) client_negotiation.cc:240] Negotiation successful
0809 22:56:38.623785 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":801,"mutex_wait_us":185,"thread_start_us":426,"threads_started":1}
I20260809 22:56:38.624226 8660 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "692fbd11681248d18ddf09fac28423fd" candidate_uuid: "a8594d32d7724c3b831d36b48c70ba60" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c988eacb729c44a0bdd072dc42707c58" is_pre_election: true
I20260809 22:56:38.624828 8660 raft_consensus.cc:2454] T 692fbd11681248d18ddf09fac28423fd P c988eacb729c44a0bdd072dc42707c58 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a8594d32d7724c3b831d36b48c70ba60 in term 0.
I20260809 22:56:38.625214 8842 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "692fbd11681248d18ddf09fac28423fd" candidate_uuid: "a8594d32d7724c3b831d36b48c70ba60" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0022f56469854dd5b9783b4e88420ef5" is_pre_election: true
I20260809 22:56:38.625563 8937 leader_election.cc:302] T 692fbd11681248d18ddf09fac28423fd P a8594d32d7724c3b831d36b48c70ba60 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a8594d32d7724c3b831d36b48c70ba60, c988eacb729c44a0bdd072dc42707c58; no voters:
I20260809 22:56:38.625739 8842 raft_consensus.cc:2454] T 692fbd11681248d18ddf09fac28423fd P 0022f56469854dd5b9783b4e88420ef5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a8594d32d7724c3b831d36b48c70ba60 in term 0.
I20260809 22:56:38.626013 9121 raft_consensus.cc:2781] T 692fbd11681248d18ddf09fac28423fd P a8594d32d7724c3b831d36b48c70ba60 [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260809 22:56:38.626053 8725 tablet.cc:2369] T 692fbd11681248d18ddf09fac28423fd P c988eacb729c44a0bdd072dc42707c58: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 22:56:38.626194 9121 raft_consensus.cc:482] T 692fbd11681248d18ddf09fac28423fd P a8594d32d7724c3b831d36b48c70ba60 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 22:56:38.626308 9121 raft_consensus.cc:3037] T 692fbd11681248d18ddf09fac28423fd P a8594d32d7724c3b831d36b48c70ba60 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:56:38.628556 9121 raft_consensus.cc:504] T 692fbd11681248d18ddf09fac28423fd P a8594d32d7724c3b831d36b48c70ba60 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c988eacb729c44a0bdd072dc42707c58" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 40825 } } peers { permanent_uuid: "0022f56469854dd5b9783b4e88420ef5" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 33009 } } peers { permanent_uuid: "a8594d32d7724c3b831d36b48c70ba60" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 46709 } }
I20260809 22:56:38.629202 9121 leader_election.cc:288] T 692fbd11681248d18ddf09fac28423fd P a8594d32d7724c3b831d36b48c70ba60 [CANDIDATE]: Term 1 election: Requested vote from peers c988eacb729c44a0bdd072dc42707c58 (127.15.113.129:40825), 0022f56469854dd5b9783b4e88420ef5 (127.15.113.130:33009)
I20260809 22:56:38.629534 8660 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "692fbd11681248d18ddf09fac28423fd" candidate_uuid: "a8594d32d7724c3b831d36b48c70ba60" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c988eacb729c44a0bdd072dc42707c58"
I20260809 22:56:38.629667 8842 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "692fbd11681248d18ddf09fac28423fd" candidate_uuid: "a8594d32d7724c3b831d36b48c70ba60" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0022f56469854dd5b9783b4e88420ef5"
I20260809 22:56:38.629727 8660 raft_consensus.cc:3037] T 692fbd11681248d18ddf09fac28423fd P c988eacb729c44a0bdd072dc42707c58 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:56:38.629859 8842 raft_consensus.cc:3037] T 692fbd11681248d18ddf09fac28423fd P 0022f56469854dd5b9783b4e88420ef5 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:56:38.631830 8660 raft_consensus.cc:2454] T 692fbd11681248d18ddf09fac28423fd P c988eacb729c44a0bdd072dc42707c58 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a8594d32d7724c3b831d36b48c70ba60 in term 1.
I20260809 22:56:38.631997 8842 raft_consensus.cc:2454] T 692fbd11681248d18ddf09fac28423fd P 0022f56469854dd5b9783b4e88420ef5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a8594d32d7724c3b831d36b48c70ba60 in term 1.
I20260809 22:56:38.632186 8937 leader_election.cc:302] T 692fbd11681248d18ddf09fac28423fd P a8594d32d7724c3b831d36b48c70ba60 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a8594d32d7724c3b831d36b48c70ba60, c988eacb729c44a0bdd072dc42707c58; no voters:
I20260809 22:56:38.632450 9121 raft_consensus.cc:2781] T 692fbd11681248d18ddf09fac28423fd P a8594d32d7724c3b831d36b48c70ba60 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 22:56:38.632843 9121 raft_consensus.cc:686] T 692fbd11681248d18ddf09fac28423fd P a8594d32d7724c3b831d36b48c70ba60 [term 1 LEADER]: Becoming Leader. State: Replica: a8594d32d7724c3b831d36b48c70ba60, State: Running, Role: LEADER
I20260809 22:56:38.633324 9121 consensus_queue.cc:237] T 692fbd11681248d18ddf09fac28423fd P a8594d32d7724c3b831d36b48c70ba60 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c988eacb729c44a0bdd072dc42707c58" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 40825 } } peers { permanent_uuid: "0022f56469854dd5b9783b4e88420ef5" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 33009 } } peers { permanent_uuid: "a8594d32d7724c3b831d36b48c70ba60" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 46709 } }
I20260809 22:56:38.637797 8458 catalog_manager.cc:5495] T 692fbd11681248d18ddf09fac28423fd P a8594d32d7724c3b831d36b48c70ba60 reported cstate change: term changed from 0 to 1, leader changed from <none> to a8594d32d7724c3b831d36b48c70ba60 (127.15.113.131). New cstate: current_term: 1 leader_uuid: "a8594d32d7724c3b831d36b48c70ba60" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c988eacb729c44a0bdd072dc42707c58" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 40825 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0022f56469854dd5b9783b4e88420ef5" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 33009 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a8594d32d7724c3b831d36b48c70ba60" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 46709 } health_report { overall_health: HEALTHY } } }
I20260809 22:56:38.681789 9109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:56:38.676558 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44308 (local address 127.15.113.131:46709)
0809 22:56:38.676757 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0809 22:56:38.676762 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:56:38.676783 (+ 21us) server_negotiation.cc:407] Connection header received
0809 22:56:38.676934 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:56:38.676938 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:56:38.677030 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:56:38.677173 (+ 143us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 22:56:38.677539 (+ 366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:56:38.678268 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:56:38.679636 (+ 1368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:56:38.680024 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:56:38.680142 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:56:38.680627 (+ 485us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 22:56:38.681173 (+ 546us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 22:56:38.681306 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0809 22:56:38.681407 (+ 101us) server_negotiation.cc:299] Negotiation successful
0809 22:56:38.681487 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260809 22:56:38.689515 8842 raft_consensus.cc:1261] T 692fbd11681248d18ddf09fac28423fd P 0022f56469854dd5b9783b4e88420ef5 [term 1 FOLLOWER]: Refusing update from remote peer a8594d32d7724c3b831d36b48c70ba60: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 22:56:38.689533 8660 raft_consensus.cc:1261] T 692fbd11681248d18ddf09fac28423fd P c988eacb729c44a0bdd072dc42707c58 [term 1 FOLLOWER]: Refusing update from remote peer a8594d32d7724c3b831d36b48c70ba60: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 22:56:38.690397 9123 consensus_queue.cc:1035] T 692fbd11681248d18ddf09fac28423fd P a8594d32d7724c3b831d36b48c70ba60 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c988eacb729c44a0bdd072dc42707c58" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 40825 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 22:56:38.690738 9121 consensus_queue.cc:1035] T 692fbd11681248d18ddf09fac28423fd P a8594d32d7724c3b831d36b48c70ba60 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0022f56469854dd5b9783b4e88420ef5" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 33009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 22:56:38.703590 9132 mvcc.cc:205] Tried to move back new op lower bound from 7316751149822754816 to 7316751149605507072. Current Snapshot: MvccSnapshot[applied={T|T < 7316751149822754816}]
I20260809 22:56:38.716382 8458 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59648:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 22:56:38.716671 8458 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59648:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 22:56:38.719914 8458 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0f3abae906c2437aaf43e0bef2ad018c: Sending DeleteTablet for 3 replicas of tablet 692fbd11681248d18ddf09fac28423fd
I20260809 22:56:38.721002 9004 tablet_service.cc:1514] Processing DeleteTablet for tablet 692fbd11681248d18ddf09fac28423fd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:56:38 UTC) from {username='kudu', principal='kudu/127.15.113.190@KRBTEST.COM'} at 127.0.0.1:44304
I20260809 22:56:38.720999 8630 tablet_service.cc:1514] Processing DeleteTablet for tablet 692fbd11681248d18ddf09fac28423fd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:56:38 UTC) from {username='kudu', principal='kudu/127.15.113.190@KRBTEST.COM'} at 127.0.0.1:36106
I20260809 22:56:38.721014 8813 tablet_service.cc:1514] Processing DeleteTablet for tablet 692fbd11681248d18ddf09fac28423fd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:56:38 UTC) from {username='kudu', principal='kudu/127.15.113.190@KRBTEST.COM'} at 127.0.0.1:44080
I20260809 22:56:38.722424 15814 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskItYPJR/build/asan/bin/kudu pid 8520
I20260809 22:56:38.741106 9103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:56:38.723302 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39700 (local address 127.15.113.129:40825)
0809 22:56:38.723507 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0809 22:56:38.723512 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:56:38.723531 (+ 19us) server_negotiation.cc:407] Connection header received
0809 22:56:38.723688 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:56:38.723693 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:56:38.723776 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:56:38.723867 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:56:38.725781 (+ 1914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:56:38.727091 (+ 1310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:56:38.729087 (+ 1996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:56:38.729797 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:56:38.729901 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:56:38.733718 (+ 3817us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:56:38.733735 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:56:38.733757 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:56:38.733802 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:56:38.737873 (+ 4071us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:56:38.738775 (+ 902us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:56:38.738786 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:56:38.738791 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:56:38.738990 (+ 199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:56:38.739382 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:56:38.739387 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:56:38.739391 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:56:38.739896 (+ 505us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:56:38.740197 (+ 301us) server_negotiation.cc:1035] Waiting for connection context
0809 22:56:38.740466 (+ 269us) server_negotiation.cc:299] Negotiation successful
0809 22:56:38.740678 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260809 22:56:38.742084 8715 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:56:38.748917 9143 tablet_replica.cc:330] T 692fbd11681248d18ddf09fac28423fd P 0022f56469854dd5b9783b4e88420ef5: stopping tablet replica
I20260809 22:56:38.750090 9143 raft_consensus.cc:2229] T 692fbd11681248d18ddf09fac28423fd P 0022f56469854dd5b9783b4e88420ef5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 22:56:38.751041 9143 raft_consensus.cc:2258] T 692fbd11681248d18ddf09fac28423fd P 0022f56469854dd5b9783b4e88420ef5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 22:56:38.752442 9144 tablet_replica.cc:330] T 692fbd11681248d18ddf09fac28423fd P a8594d32d7724c3b831d36b48c70ba60: stopping tablet replica
I20260809 22:56:38.752863 9144 raft_consensus.cc:2229] T 692fbd11681248d18ddf09fac28423fd P a8594d32d7724c3b831d36b48c70ba60 [term 1 LEADER]: Raft consensus shutting down.
I20260809 22:56:38.754179 9143 ts_tablet_manager.cc:1905] T 692fbd11681248d18ddf09fac28423fd P 0022f56469854dd5b9783b4e88420ef5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 22:56:38.754889 9144 raft_consensus.cc:2258] T 692fbd11681248d18ddf09fac28423fd P a8594d32d7724c3b831d36b48c70ba60 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 22:56:38.757979 9144 ts_tablet_manager.cc:1905] T 692fbd11681248d18ddf09fac28423fd P a8594d32d7724c3b831d36b48c70ba60: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 22:56:38.761991 9143 ts_tablet_manager.cc:1918] T 692fbd11681248d18ddf09fac28423fd P 0022f56469854dd5b9783b4e88420ef5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 22:56:38.762280 9143 log.cc:1196] T 692fbd11681248d18ddf09fac28423fd P 0022f56469854dd5b9783b4e88420ef5: Deleting WAL directory at /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315935617919-15814-0/minicluster-data/ts-1/wal/wals/692fbd11681248d18ddf09fac28423fd
I20260809 22:56:38.763773 9143 ts_tablet_manager.cc:1939] T 692fbd11681248d18ddf09fac28423fd P 0022f56469854dd5b9783b4e88420ef5: Deleting consensus metadata
I20260809 22:56:38.766076 8436 catalog_manager.cc:4841] TS 0022f56469854dd5b9783b4e88420ef5 (127.15.113.130:33009): tablet 692fbd11681248d18ddf09fac28423fd (table test-table [id=5d0fdb2ec6c9473da9eb25d71c582a81]) successfully deleted
I20260809 22:56:38.768711 9144 ts_tablet_manager.cc:1918] T 692fbd11681248d18ddf09fac28423fd P a8594d32d7724c3b831d36b48c70ba60: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 22:56:38.768872 9144 log.cc:1196] T 692fbd11681248d18ddf09fac28423fd P a8594d32d7724c3b831d36b48c70ba60: Deleting WAL directory at /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315935617919-15814-0/minicluster-data/ts-2/wal/wals/692fbd11681248d18ddf09fac28423fd
I20260809 22:56:38.769346 9144 ts_tablet_manager.cc:1939] T 692fbd11681248d18ddf09fac28423fd P a8594d32d7724c3b831d36b48c70ba60: Deleting consensus metadata
I20260809 22:56:38.770815 8438 catalog_manager.cc:4841] TS a8594d32d7724c3b831d36b48c70ba60 (127.15.113.131:46709): tablet 692fbd11681248d18ddf09fac28423fd (table test-table [id=5d0fdb2ec6c9473da9eb25d71c582a81]) successfully deleted
I20260809 22:56:38.914431 9145 tablet_replica.cc:330] T 692fbd11681248d18ddf09fac28423fd P c988eacb729c44a0bdd072dc42707c58: stopping tablet replica
I20260809 22:56:38.914860 9145 raft_consensus.cc:2229] T 692fbd11681248d18ddf09fac28423fd P c988eacb729c44a0bdd072dc42707c58 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 22:56:38.915297 9145 raft_consensus.cc:2258] T 692fbd11681248d18ddf09fac28423fd P c988eacb729c44a0bdd072dc42707c58 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 22:56:38.916448 15814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskItYPJR/build/asan/bin/kudu with pid 8520
W20260809 22:56:38.939545 8437 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.15.113.129:40825 (error 108)
I20260809 22:56:38.939654 15814 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskItYPJR/build/asan/bin/kudu pid 8729
I20260809 22:56:38.947240 9149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:56:38.940964 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.129:40825 (local address 127.0.0.1:39702)
0809 22:56:38.946681 (+ 5717us) negotiation.cc:107] Waiting for socket to connect
0809 22:56:38.946911 (+ 230us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.113.129:40825: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":5549,"thread_start_us":5443,"threads_started":1}
W20260809 22:56:38.947846 8437 catalog_manager.cc:4584] TS c988eacb729c44a0bdd072dc42707c58 (127.15.113.129:40825): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 692fbd11681248d18ddf09fac28423fd: Network error: Client connection negotiation failed: client connection to 127.15.113.129:40825: connect: Connection refused (error 111)
I20260809 22:56:38.964377 9105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:56:38.940426 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41190 (local address 127.15.113.130:33009)
0809 22:56:38.940622 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0809 22:56:38.940629 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:56:38.940963 (+ 334us) server_negotiation.cc:407] Connection header received
0809 22:56:38.941044 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:56:38.941050 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:56:38.941143 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:56:38.941217 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:56:38.948332 (+ 7115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:56:38.949207 (+ 875us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:56:38.951133 (+ 1926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:56:38.951516 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:56:38.951607 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:56:38.958848 (+ 7241us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:56:38.958864 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:56:38.958879 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:56:38.958914 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:56:38.961724 (+ 2810us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:56:38.962519 (+ 795us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:56:38.962525 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:56:38.962527 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:56:38.962641 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:56:38.963026 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:56:38.963030 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:56:38.963032 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:56:38.963391 (+ 359us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:56:38.963603 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0809 22:56:38.963950 (+ 347us) server_negotiation.cc:299] Negotiation successful
0809 22:56:38.964112 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260809 22:56:38.965063 8903 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:56:39.001796 9149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:56:39.001235 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.129:40825 (local address 127.0.0.1:39714)
0809 22:56:39.001492 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0809 22:56:39.001619 (+ 127us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.113.129:40825: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":83}
I20260809 22:56:39.003038 9149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:56:39.002613 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.129:40825 (local address 127.0.0.1:39730)
0809 22:56:39.002789 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0809 22:56:39.002879 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.113.129:40825: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260809 22:56:39.053035 15814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskItYPJR/build/asan/bin/kudu with pid 8729
I20260809 22:56:39.063941 9149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:56:39.063308 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.129:40825 (local address 127.0.0.1:39746)
0809 22:56:39.063641 (+ 333us) negotiation.cc:107] Waiting for socket to connect
0809 22:56:39.063754 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.113.129:40825: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":187}
I20260809 22:56:39.065209 9149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:56:39.064790 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.129:40825 (local address 127.0.0.1:39760)
0809 22:56:39.064963 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0809 22:56:39.065054 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.113.129:40825: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260809 22:56:39.081059 15814 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskItYPJR/build/asan/bin/kudu pid 8913
I20260809 22:56:39.095043 9109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:56:39.081789 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44164 (local address 127.15.113.131:46709)
0809 22:56:39.081960 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0809 22:56:39.081965 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:56:39.082071 (+ 106us) server_negotiation.cc:407] Connection header received
0809 22:56:39.082276 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:56:39.082282 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:56:39.082354 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:56:39.082441 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:56:39.083791 (+ 1350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:56:39.084592 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:56:39.085957 (+ 1365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:56:39.086409 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:56:39.086545 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:56:39.089243 (+ 2698us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:56:39.089256 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:56:39.089272 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:56:39.089312 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:56:39.092335 (+ 3023us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:56:39.092975 (+ 640us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:56:39.092984 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:56:39.092988 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:56:39.093144 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:56:39.093515 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:56:39.093518 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:56:39.093520 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:56:39.093864 (+ 344us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:56:39.094175 (+ 311us) server_negotiation.cc:1035] Waiting for connection context
0809 22:56:39.094508 (+ 333us) server_negotiation.cc:299] Negotiation successful
0809 22:56:39.094724 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260809 22:56:39.095714 9083 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:56:39.145931 9149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:56:39.145372 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.129:40825 (local address 127.0.0.1:39776)
0809 22:56:39.145611 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0809 22:56:39.145742 (+ 131us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.113.129:40825: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
I20260809 22:56:39.147038 9149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:56:39.146601 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.129:40825 (local address 127.0.0.1:39782)
0809 22:56:39.146800 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0809 22:56:39.146886 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.113.129:40825: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101}
I20260809 22:56:39.217515 15814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskItYPJR/build/asan/bin/kudu with pid 8913
I20260809 22:56:39.242488 15814 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskItYPJR/build/asan/bin/kudu pid 8414
I20260809 22:56:39.243541 8513 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:56:39.354300 9149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:56:39.353799 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.129:40825 (local address 127.0.0.1:39790)
0809 22:56:39.354011 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0809 22:56:39.354108 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.113.129:40825: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
I20260809 22:56:39.354302 15814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskItYPJR/build/asan/bin/kudu with pid 8414
I20260809 22:56:59.379482 15814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskItYPJR/build/asan/bin/kudu
/tmp/dist-test-taskItYPJR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315935617919-15814-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315935617919-15814-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315935617919-15814-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315935617919-15814-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.113.190:39625
--webserver_interface=127.15.113.190
--webserver_port=36581
--keytab_file=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.190.keytab
--principal=kudu/127.15.113.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.113.148:46195
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.113.190:39625
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315935617919-15814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315935617919-15814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315935617919-15814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:56:59.714461 9154 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:56:59.714751 9154 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:56:59.714880 9154 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:56:59.721912 9154 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260809 22:56:59.722005 9154 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260809 22:56:59.722065 9154 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260809 22:56:59.722136 9154 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 22:56:59.722222 9154 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:56:59.722314 9154 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:56:59.722375 9154 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 22:56:59.722458 9154 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 22:56:59.731932 9154 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.113.148:46195
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315935617919-15814-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315935617919-15814-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.15.113.190:39625
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.190.keytab
--principal=kudu/127.15.113.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.113.190:39625
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315935617919-15814-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.113.190
--webserver_port=36581
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315935617919-15814-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:56:59.733167 9154 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:56:59.734445 9154 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:56:59.745930 9164 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:56:59.746025 9161 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:56:59.745975 9162 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:56:59.746477 9154 server_base.cc:1034] running on GCE node
I20260809 22:56:59.747316 9154 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:56:59.748605 9154 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:56:59.749819 9154 hybrid_clock.cc:648] HybridClock initialized: now 1786316219749748 us; error 40 us; skew 500 ppm
I20260809 22:56:59.750149 9154 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:56:59 dist-test-slave-ncp9 krb5kdc[7625](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316219, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.113.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:56:59.757439 9154 init.cc:377] Logged in from keytab as kudu/127.15.113.190@KRBTEST.COM (short username kudu)
I20260809 22:56:59.758769 9154 webserver.cc:460] Webserver started at http://127.15.113.190:36581/ using document root <none> and password file <none>
I20260809 22:56:59.759420 9154 fs_manager.cc:359] Metadata directory not provided
I20260809 22:56:59.759521 9154 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:56:59.764664 9154 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.001s sys 0.001s
I20260809 22:56:59.767280 9172 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:56:59.768502 9154 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260809 22:56:59.768663 9154 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315935617919-15814-0/minicluster-data/master-0/data,/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315935617919-15814-0/minicluster-data/master-0/wal
uuid: "0f3abae906c2437aaf43e0bef2ad018c"
format_stamp: "Formatted at 2026-08-09 22:56:13 on dist-test-slave-ncp9"
server_key: "38c512142111980ce2d84baa62069423"
server_key_iv: "bd6328eedf7756daae78341d56af6aff"
server_key_version: "encryptionkey@0"
I20260809 22:56:59.769598 9154 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315935617919-15814-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315935617919-15814-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315935617919-15814-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:56:59.794739 9154 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:56:59.798629 9154 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:56:59.798887 9154 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:56:59.815948 9154 rpc_server.cc:307] RPC server started. Bound to: 127.15.113.190:39625
I20260809 22:56:59.815984 9240 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.113.190:39625 every 8 connection(s)
I20260809 22:56:59.817297 9154 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315935617919-15814-0/minicluster-data/master-0/data/info.pb
I20260809 22:56:59.823855 9241 sys_catalog.cc:263] Verifying existing consensus state
I20260809 22:56:59.826969 9241 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0f3abae906c2437aaf43e0bef2ad018c: Bootstrap starting.
I20260809 22:56:59.827569 15814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskItYPJR/build/asan/bin/kudu as pid 9154
I20260809 22:56:59.828400 15814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskItYPJR/build/asan/bin/kudu
/tmp/dist-test-taskItYPJR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315935617919-15814-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315935617919-15814-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315935617919-15814-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315935617919-15814-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.113.129:40825
--local_ip_for_outbound_sockets=127.15.113.129
--tserver_master_addrs=127.15.113.190:39625
--webserver_port=34871
--webserver_interface=127.15.113.129
--keytab_file=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.129.keytab
--principal=kudu/127.15.113.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.113.148:46195
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315935617919-15814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315935617919-15814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315935617919-15814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 22:56:59.846637 9241 log.cc:824] T 00000000000000000000000000000000 P 0f3abae906c2437aaf43e0bef2ad018c: Log is configured to *not* fsync() on all Append() calls
I20260809 22:56:59.862617 9241 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0f3abae906c2437aaf43e0bef2ad018c: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 22:56:59.863104 9241 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0f3abae906c2437aaf43e0bef2ad018c: Bootstrap complete.
I20260809 22:56:59.872169 9241 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0f3abae906c2437aaf43e0bef2ad018c [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f3abae906c2437aaf43e0bef2ad018c" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 39625 } }
I20260809 22:56:59.873185 9241 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0f3abae906c2437aaf43e0bef2ad018c [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0f3abae906c2437aaf43e0bef2ad018c, State: Initialized, Role: FOLLOWER
I20260809 22:56:59.873834 9241 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0f3abae906c2437aaf43e0bef2ad018c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f3abae906c2437aaf43e0bef2ad018c" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 39625 } }
I20260809 22:56:59.874064 9241 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0f3abae906c2437aaf43e0bef2ad018c [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 22:56:59.874181 9241 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0f3abae906c2437aaf43e0bef2ad018c [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 22:56:59.874348 9241 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0f3abae906c2437aaf43e0bef2ad018c [term 2 FOLLOWER]: Advancing to term 3
I20260809 22:56:59.878057 9241 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0f3abae906c2437aaf43e0bef2ad018c [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f3abae906c2437aaf43e0bef2ad018c" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 39625 } }
I20260809 22:56:59.878527 9241 leader_election.cc:302] T 00000000000000000000000000000000 P 0f3abae906c2437aaf43e0bef2ad018c [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0f3abae906c2437aaf43e0bef2ad018c; no voters:
I20260809 22:56:59.879150 9241 leader_election.cc:288] T 00000000000000000000000000000000 P 0f3abae906c2437aaf43e0bef2ad018c [CANDIDATE]: Term 3 election: Requested vote from peers
I20260809 22:56:59.879352 9246 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0f3abae906c2437aaf43e0bef2ad018c [term 3 FOLLOWER]: Leader election won for term 3
I20260809 22:56:59.880357 9246 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0f3abae906c2437aaf43e0bef2ad018c [term 3 LEADER]: Becoming Leader. State: Replica: 0f3abae906c2437aaf43e0bef2ad018c, State: Running, Role: LEADER
I20260809 22:56:59.881155 9241 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0f3abae906c2437aaf43e0bef2ad018c [sys.catalog]: configured and running, proceeding with master startup.
I20260809 22:56:59.881079 9246 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0f3abae906c2437aaf43e0bef2ad018c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f3abae906c2437aaf43e0bef2ad018c" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 39625 } }
I20260809 22:56:59.885182 9248 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0f3abae906c2437aaf43e0bef2ad018c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "0f3abae906c2437aaf43e0bef2ad018c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f3abae906c2437aaf43e0bef2ad018c" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 39625 } } }
I20260809 22:56:59.885211 9249 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0f3abae906c2437aaf43e0bef2ad018c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0f3abae906c2437aaf43e0bef2ad018c. Latest consensus state: current_term: 3 leader_uuid: "0f3abae906c2437aaf43e0bef2ad018c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f3abae906c2437aaf43e0bef2ad018c" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 39625 } } }
I20260809 22:56:59.885649 9249 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0f3abae906c2437aaf43e0bef2ad018c [sys.catalog]: This master's current role is: LEADER
I20260809 22:56:59.885656 9248 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0f3abae906c2437aaf43e0bef2ad018c [sys.catalog]: This master's current role is: LEADER
I20260809 22:56:59.886605 9256 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 22:56:59.903962 9256 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 22:56:59.905557 9256 catalog_manager.cc:1259] Loaded cluster ID: 0589e66c0b8b476bb039a06f3f16b159
I20260809 22:56:59.905727 9256 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 22:56:59.909852 9256 catalog_manager.cc:1504] Loading token signing keys...
I20260809 22:56:59.911722 9256 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 0f3abae906c2437aaf43e0bef2ad018c: Loaded TSK: 0
I20260809 22:56:59.911952 9256 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 0f3abae906c2437aaf43e0bef2ad018c: Loaded TSK: 1
F20260809 22:56:59.913208 9256 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7fc172acd9ad google::LogMessage::Fail() at ??:0
@ 0x7fc172ad088a google::LogMessage::SendToLog() at ??:0
@ 0x7fc172acd4de google::LogMessage::Flush() at ??:0
@ 0x7fc172acea19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7fc1768317d2 kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7fc17682073e kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7fc1884fc1c0 kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7fc1884fb783 kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7fc1885872a7 kudu::master::CatalogManager::PrepareForLeadershipTask()::$_10::operator()() at ??:0
@ 0x7fc1885870c4 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x7fc1885d15e9 std::function<>::operator()() at ??:0
@ 0x7fc1884ffcdd kudu::master::CatalogManager::PrepareForLeadershipTask()::$_6::operator()() at ??:0
@ 0x7fc1884fdc36 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7fc188585610 kudu::master::CatalogManager::ElectedAsLeaderCb()::$_2::operator()() at ??:0
@ 0x7fc188585439 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
I20260809 22:56:59.972385 9274 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
@ 0x7fc175719345 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7fc17571c91c kudu::ThreadPool::CreateThread()::$_2::operator()() at ??:0
@ 0x7fc17571c799 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7fc1756f7eaf kudu::Thread::SuperviseThread() at ??:0
@ 0x7fc1789d16db start_thread at ??:0
@ 0x7fc17196761f clone at ??:0
W20260809 22:57:00.233467 9243 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:57:00.233733 9243 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:57:00.233853 9243 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:57:00.240841 9243 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:57:00.240962 9243 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:57:00.241127 9243 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.113.129
I20260809 22:57:00.250381 9243 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.113.148:46195
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315935617919-15814-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315935617919-15814-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.129.keytab
--principal=kudu/127.15.113.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.113.129:40825
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315935617919-15814-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.113.129
--webserver_port=34871
--tserver_master_addrs=127.15.113.190:39625
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.113.129
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315935617919-15814-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:57:00.251509 9243 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:57:00.252795 9243 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:57:00.265234 9304 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:57:00.265317 9305 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:57:00.265285 9308 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:57:00.266207 9243 server_base.cc:1034] running on GCE node
I20260809 22:57:00.266780 9243 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:57:00.267725 9243 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:57:00.268944 9243 hybrid_clock.cc:648] HybridClock initialized: now 1786316220268870 us; error 40 us; skew 500 ppm
I20260809 22:57:00.269212 9243 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:57:00 dist-test-slave-ncp9 krb5kdc[7625](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316220, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.113.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:57:00.272053 9243 init.cc:377] Logged in from keytab as kudu/127.15.113.129@KRBTEST.COM (short username kudu)
I20260809 22:57:00.273365 9243 webserver.cc:460] Webserver started at http://127.15.113.129:34871/ using document root <none> and password file <none>
I20260809 22:57:00.273977 9243 fs_manager.cc:359] Metadata directory not provided
I20260809 22:57:00.274091 9243 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:57:00.279158 9243 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260809 22:57:00.281857 9320 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:57:00.283114 9243 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 22:57:00.283245 9243 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315935617919-15814-0/minicluster-data/ts-0/data,/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315935617919-15814-0/minicluster-data/ts-0/wal
uuid: "c988eacb729c44a0bdd072dc42707c58"
format_stamp: "Formatted at 2026-08-09 22:56:14 on dist-test-slave-ncp9"
server_key: "6791de00355b98f99ffb6e181829269d"
server_key_iv: "4880d044c42044fe3964d38b4a81c2dc"
server_key_version: "encryptionkey@0"
I20260809 22:57:00.284150 9243 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315935617919-15814-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315935617919-15814-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315935617919-15814-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:57:00.299898 9243 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:57:00.303380 9243 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:57:00.303625 9243 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:57:00.304529 9243 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:57:00.306228 9328 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
I20260809 22:57:00.311395 9243 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260809 22:57:00.311491 9243 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.006s user 0.001s sys 0.000s
I20260809 22:57:00.311610 9243 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260809 22:57:00.314678 9243 ts_tablet_manager.cc:610] Registered 1 tablets
I20260809 22:57:00.314764 9243 ts_tablet_manager.cc:589] Time spent register tablets: real 0.003s user 0.002s sys 0.000s
I20260809 22:57:00.315044 9328 tablet_bootstrap.cc:492] T 692fbd11681248d18ddf09fac28423fd P c988eacb729c44a0bdd072dc42707c58: Bootstrap starting.
I20260809 22:57:00.322721 9328 log.cc:824] T 692fbd11681248d18ddf09fac28423fd P c988eacb729c44a0bdd072dc42707c58: Log is configured to *not* fsync() on all Append() calls
I20260809 22:57:00.336139 9328 tablet_bootstrap.cc:492] T 692fbd11681248d18ddf09fac28423fd P c988eacb729c44a0bdd072dc42707c58: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 22:57:00.337056 9328 tablet_bootstrap.cc:492] T 692fbd11681248d18ddf09fac28423fd P c988eacb729c44a0bdd072dc42707c58: Bootstrap complete.
I20260809 22:57:00.337621 9328 ts_tablet_manager.cc:1397] T 692fbd11681248d18ddf09fac28423fd P c988eacb729c44a0bdd072dc42707c58: Time spent bootstrapping tablet: real 0.023s user 0.017s sys 0.004s
I20260809 22:57:00.346383 9328 raft_consensus.cc:348] T 692fbd11681248d18ddf09fac28423fd P c988eacb729c44a0bdd072dc42707c58 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c988eacb729c44a0bdd072dc42707c58" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 40825 } } peers { permanent_uuid: "0022f56469854dd5b9783b4e88420ef5" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 33009 } } peers { permanent_uuid: "a8594d32d7724c3b831d36b48c70ba60" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 46709 } }
I20260809 22:57:00.347782 9328 raft_consensus.cc:729] T 692fbd11681248d18ddf09fac28423fd P c988eacb729c44a0bdd072dc42707c58 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: c988eacb729c44a0bdd072dc42707c58, State: Initialized, Role: FOLLOWER
I20260809 22:57:00.348711 9328 consensus_queue.cc:260] T 692fbd11681248d18ddf09fac28423fd P c988eacb729c44a0bdd072dc42707c58 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c988eacb729c44a0bdd072dc42707c58" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 40825 } } peers { permanent_uuid: "0022f56469854dd5b9783b4e88420ef5" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 33009 } } peers { permanent_uuid: "a8594d32d7724c3b831d36b48c70ba60" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 46709 } }
I20260809 22:57:00.350436 9328 ts_tablet_manager.cc:1428] T 692fbd11681248d18ddf09fac28423fd P c988eacb729c44a0bdd072dc42707c58: Time spent starting tablet: real 0.013s user 0.004s sys 0.008s
I20260809 22:57:00.355281 9243 rpc_server.cc:307] RPC server started. Bound to: 127.15.113.129:40825
I20260809 22:57:00.355334 9483 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.113.129:40825 every 8 connection(s)
I20260809 22:57:00.356472 9243 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315935617919-15814-0/minicluster-data/ts-0/data/info.pb
I20260809 22:57:00.358668 15814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskItYPJR/build/asan/bin/kudu as pid 9243
I20260809 22:57:00.359289 15814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskItYPJR/build/asan/bin/kudu
/tmp/dist-test-taskItYPJR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315935617919-15814-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315935617919-15814-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315935617919-15814-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315935617919-15814-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.113.130:33009
--local_ip_for_outbound_sockets=127.15.113.130
--tserver_master_addrs=127.15.113.190:39625
--webserver_port=45625
--webserver_interface=127.15.113.130
--keytab_file=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.130.keytab
--principal=kudu/127.15.113.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.113.148:46195
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315935617919-15814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315935617919-15814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315935617919-15814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 22:57:00.361357 9489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:57:00.359089 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:39625 (local address 127.15.113.129:39395)
0809 22:57:00.359932 (+ 843us) negotiation.cc:107] Waiting for socket to connect
0809 22:57:00.360191 (+ 259us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.113.190:39625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":553,"thread_start_us":333,"threads_started":1}
W20260809 22:57:00.361986 9487 heartbeater.cc:644] Failed to heartbeat to 127.15.113.190:39625 (0 consecutive failures): Network error: Failed to ping master at 127.15.113.190:39625: Client connection negotiation failed: client connection to 127.15.113.190:39625: connect: Connection refused (error 111)
I20260809 22:57:00.362811 9489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:57:00.362474 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:39625 (local address 127.15.113.129:51603)
0809 22:57:00.362619 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0809 22:57:00.362698 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.113.190:39625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260809 22:57:00.363689 9489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:57:00.363318 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:39625 (local address 127.15.113.129:46041)
0809 22:57:00.363451 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0809 22:57:00.363530 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.113.190:39625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260809 22:57:00.363912 9487 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260809 22:57:00.688699 9492 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:57:00.688997 9492 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:57:00.689141 9492 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:57:00.696084 9492 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:57:00.696193 9492 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:57:00.696342 9492 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.113.130
I20260809 22:57:00.705803 9492 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.113.148:46195
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315935617919-15814-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315935617919-15814-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.130.keytab
--principal=kudu/127.15.113.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.113.130:33009
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315935617919-15814-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.113.130
--webserver_port=45625
--tserver_master_addrs=127.15.113.190:39625
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.113.130
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315935617919-15814-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:57:00.707198 9492 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:57:00.708482 9492 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:57:00.721055 9503 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:57:00.721017 9502 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:57:00.721087 9505 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:57:00.721840 9492 server_base.cc:1034] running on GCE node
I20260809 22:57:00.722373 9492 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:57:00.723208 9492 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:57:00.724395 9492 hybrid_clock.cc:648] HybridClock initialized: now 1786316220724353 us; error 30 us; skew 500 ppm
I20260809 22:57:00.724685 9492 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:57:00 dist-test-slave-ncp9 krb5kdc[7625](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316220, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.113.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:57:00.727560 9492 init.cc:377] Logged in from keytab as kudu/127.15.113.130@KRBTEST.COM (short username kudu)
I20260809 22:57:00.728857 9492 webserver.cc:460] Webserver started at http://127.15.113.130:45625/ using document root <none> and password file <none>
I20260809 22:57:00.729504 9492 fs_manager.cc:359] Metadata directory not provided
I20260809 22:57:00.729617 9492 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:57:00.734591 9492 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260809 22:57:00.737152 9514 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:57:00.738382 9492 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 22:57:00.738508 9492 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315935617919-15814-0/minicluster-data/ts-1/data,/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315935617919-15814-0/minicluster-data/ts-1/wal
uuid: "0022f56469854dd5b9783b4e88420ef5"
format_stamp: "Formatted at 2026-08-09 22:56:14 on dist-test-slave-ncp9"
server_key: "9a840933e2926fdf96995a968373eb05"
server_key_iv: "b62d759303acd0b00a4efd1623e5c932"
server_key_version: "encryptionkey@0"
I20260809 22:57:00.739410 9492 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315935617919-15814-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315935617919-15814-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315935617919-15814-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:57:00.754668 9492 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:57:00.758250 9492 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:57:00.758453 9492 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:57:00.759294 9492 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:57:00.760493 9492 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:57:00.760574 9492 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:57:00.760648 9492 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:57:00.760705 9492 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:57:00.790293 9492 rpc_server.cc:307] RPC server started. Bound to: 127.15.113.130:33009
I20260809 22:57:00.790359 9688 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.113.130:33009 every 8 connection(s)
I20260809 22:57:00.791704 9492 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315935617919-15814-0/minicluster-data/ts-1/data/info.pb
I20260809 22:57:00.796339 9691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:57:00.794551 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:39625 (local address 127.15.113.130:47987)
0809 22:57:00.795271 (+ 720us) negotiation.cc:107] Waiting for socket to connect
0809 22:57:00.795501 (+ 230us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.113.190:39625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":454,"thread_start_us":281,"threads_started":1}
W20260809 22:57:00.796952 9689 heartbeater.cc:644] Failed to heartbeat to 127.15.113.190:39625 (0 consecutive failures): Network error: Failed to ping master at 127.15.113.190:39625: Client connection negotiation failed: client connection to 127.15.113.190:39625: connect: Connection refused (error 111)
I20260809 22:57:00.797569 15814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskItYPJR/build/asan/bin/kudu as pid 9492
I20260809 22:57:00.797760 9691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:57:00.797462 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:39625 (local address 127.15.113.130:42963)
0809 22:57:00.797586 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0809 22:57:00.797649 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.113.190:39625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260809 22:57:00.798229 15814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskItYPJR/build/asan/bin/kudu
/tmp/dist-test-taskItYPJR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315935617919-15814-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315935617919-15814-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315935617919-15814-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315935617919-15814-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.113.131:46709
--local_ip_for_outbound_sockets=127.15.113.131
--tserver_master_addrs=127.15.113.190:39625
--webserver_port=33209
--webserver_interface=127.15.113.131
--keytab_file=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.131.keytab
--principal=kudu/127.15.113.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.113.148:46195
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315935617919-15814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315935617919-15814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315935617919-15814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 22:57:00.798566 9691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:57:00.798262 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:39625 (local address 127.15.113.130:40659)
0809 22:57:00.798395 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0809 22:57:00.798466 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.113.190:39625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260809 22:57:00.798795 9689 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260809 22:57:01.131692 9692 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:57:01.131939 9692 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:57:01.132079 9692 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:57:01.139029 9692 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:57:01.139135 9692 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:57:01.139278 9692 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.113.131
I20260809 22:57:01.148660 9692 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.113.148:46195
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315935617919-15814-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315935617919-15814-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.131.keytab
--principal=kudu/127.15.113.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.113.131:46709
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315935617919-15814-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.113.131
--webserver_port=33209
--tserver_master_addrs=127.15.113.190:39625
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.113.131
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315935617919-15814-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:57:01.149827 9692 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:57:01.151082 9692 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:57:01.163525 9701 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:57:01.163565 9699 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:57:01.163846 9698 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:57:01.164726 9692 server_base.cc:1034] running on GCE node
I20260809 22:57:01.165333 9692 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:57:01.166200 9692 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:57:01.167393 9692 hybrid_clock.cc:648] HybridClock initialized: now 1786316221167349 us; error 36 us; skew 500 ppm
I20260809 22:57:01.167675 9692 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:57:01 dist-test-slave-ncp9 krb5kdc[7625](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316221, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.113.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:57:01.170599 9692 init.cc:377] Logged in from keytab as kudu/127.15.113.131@KRBTEST.COM (short username kudu)
I20260809 22:57:01.171876 9692 webserver.cc:460] Webserver started at http://127.15.113.131:33209/ using document root <none> and password file <none>
I20260809 22:57:01.172501 9692 fs_manager.cc:359] Metadata directory not provided
I20260809 22:57:01.172616 9692 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:57:01.177739 9692 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260809 22:57:01.180287 9708 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:57:01.181605 9692 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 22:57:01.181747 9692 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315935617919-15814-0/minicluster-data/ts-2/data,/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315935617919-15814-0/minicluster-data/ts-2/wal
uuid: "a8594d32d7724c3b831d36b48c70ba60"
format_stamp: "Formatted at 2026-08-09 22:56:15 on dist-test-slave-ncp9"
server_key: "a9b5faab3611544f4151bd0358783ef9"
server_key_iv: "a82a94e30a001d1c2576a2c51c70eeee"
server_key_version: "encryptionkey@0"
I20260809 22:57:01.182718 9692 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315935617919-15814-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315935617919-15814-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315935617919-15814-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:57:01.194653 9692 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:57:01.198144 9692 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:57:01.198383 9692 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:57:01.199251 9692 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:57:01.200496 9692 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:57:01.200589 9692 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:57:01.200691 9692 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:57:01.200764 9692 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:57:01.230571 9692 rpc_server.cc:307] RPC server started. Bound to: 127.15.113.131:46709
I20260809 22:57:01.230633 9868 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.113.131:46709 every 8 connection(s)
I20260809 22:57:01.231966 9692 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315935617919-15814-0/minicluster-data/ts-2/data/info.pb
I20260809 22:57:01.236675 9871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:57:01.234817 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:39625 (local address 127.15.113.131:60737)
0809 22:57:01.235509 (+ 692us) negotiation.cc:107] Waiting for socket to connect
0809 22:57:01.235744 (+ 235us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.113.190:39625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":449,"thread_start_us":269,"threads_started":1}
I20260809 22:57:01.236999 15814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskItYPJR/build/asan/bin/kudu as pid 9692
W20260809 22:57:01.237397 9869 heartbeater.cc:644] Failed to heartbeat to 127.15.113.190:39625 (0 consecutive failures): Network error: Failed to ping master at 127.15.113.190:39625: Client connection negotiation failed: client connection to 127.15.113.190:39625: connect: Connection refused (error 111)
I20260809 22:57:01.238265 9871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:57:01.237864 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:39625 (local address 127.15.113.131:58133)
0809 22:57:01.238024 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0809 22:57:01.238117 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.113.190:39625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260809 22:57:01.239063 9871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:57:01.238761 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:39625 (local address 127.15.113.131:41863)
0809 22:57:01.238881 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0809 22:57:01.238960 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.113.190:39625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260809 22:57:01.239275 9869 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260809 22:57:01.365036 9875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:57:01.364344 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:39625 (local address 127.15.113.129:36241)
0809 22:57:01.364716 (+ 372us) negotiation.cc:107] Waiting for socket to connect
0809 22:57:01.364798 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.113.190:39625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":278,"thread_start_us":179,"threads_started":1}
I20260809 22:57:01.800117 9876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:57:01.799315 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:39625 (local address 127.15.113.130:41597)
0809 22:57:01.799763 (+ 448us) negotiation.cc:107] Waiting for socket to connect
0809 22:57:01.799874 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.113.190:39625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":341,"thread_start_us":212,"threads_started":1}
I20260809 22:57:01.993642 9877 raft_consensus.cc:482] T 692fbd11681248d18ddf09fac28423fd P c988eacb729c44a0bdd072dc42707c58 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 22:57:01.993969 9877 raft_consensus.cc:504] T 692fbd11681248d18ddf09fac28423fd P c988eacb729c44a0bdd072dc42707c58 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c988eacb729c44a0bdd072dc42707c58" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 40825 } } peers { permanent_uuid: "0022f56469854dd5b9783b4e88420ef5" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 33009 } } peers { permanent_uuid: "a8594d32d7724c3b831d36b48c70ba60" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 46709 } }
I20260809 22:57:01.996672 9877 leader_election.cc:288] T 692fbd11681248d18ddf09fac28423fd P c988eacb729c44a0bdd072dc42707c58 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0022f56469854dd5b9783b4e88420ef5 (127.15.113.130:33009), a8594d32d7724c3b831d36b48c70ba60 (127.15.113.131:46709)
Aug 09 22:57:02 dist-test-slave-ncp9 krb5kdc[7625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316220, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.113.129@KRBTEST.COM for kudu/127.15.113.131@KRBTEST.COM
Aug 09 22:57:02 dist-test-slave-ncp9 krb5kdc[7625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316220, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.113.129@KRBTEST.COM for kudu/127.15.113.130@KRBTEST.COM
I20260809 22:57:02.019687 9878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:57:01.995831 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.130:33009 (local address 127.15.113.129:42117)
0809 22:57:01.996438 (+ 607us) negotiation.cc:107] Waiting for socket to connect
0809 22:57:01.996487 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0809 22:57:01.997030 (+ 543us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:57:02.004074 (+ 7044us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:57:02.004085 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:57:02.004823 (+ 738us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:57:02.005607 (+ 784us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:57:02.005636 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:57:02.007244 (+ 1608us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:57:02.007248 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:57:02.008549 (+ 1301us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:57:02.008562 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:57:02.009551 (+ 989us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:57:02.009562 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:57:02.009810 (+ 248us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:57:02.010553 (+ 743us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:57:02.010569 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:57:02.013092 (+ 2523us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:57:02.016514 (+ 3422us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:57:02.016519 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:57:02.016524 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:57:02.016936 (+ 412us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:57:02.017356 (+ 420us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:57:02.017362 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:57:02.017366 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:57:02.017506 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:57:02.018219 (+ 713us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:57:02.018223 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:57:02.018430 (+ 207us) client_negotiation.cc:769] Sending connection context
0809 22:57:02.018692 (+ 262us) client_negotiation.cc:240] Negotiation successful
0809 22:57:02.019223 (+ 531us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":483,"mutex_wait_us":19,"spinlock_wait_cycles":30720,"thread_start_us":346,"threads_started":1}
I20260809 22:57:02.019703 9880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:57:01.998220 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.113.129:42117 (local address 127.15.113.130:33009)
0809 22:57:02.001310 (+ 3090us) server_negotiation.cc:206] Beginning negotiation
0809 22:57:02.001323 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 22:57:02.001362 (+ 39us) server_negotiation.cc:407] Connection header received
0809 22:57:02.003071 (+ 1709us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:57:02.003081 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:57:02.003568 (+ 487us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:57:02.003910 (+ 342us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:57:02.005831 (+ 1921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:57:02.007066 (+ 1235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:57:02.008684 (+ 1618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:57:02.009344 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:57:02.009508 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:57:02.013333 (+ 3825us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:57:02.013339 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:57:02.013352 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:57:02.013388 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:57:02.016385 (+ 2997us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:57:02.017063 (+ 678us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:57:02.017069 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:57:02.017074 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:57:02.017240 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:57:02.017617 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:57:02.017621 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:57:02.017624 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:57:02.018080 (+ 456us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:57:02.018276 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0809 22:57:02.018948 (+ 672us) server_negotiation.cc:299] Negotiation successful
0809 22:57:02.019333 (+ 385us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2938,"thread_start_us":2793,"threads_started":1}
I20260809 22:57:02.019695 9883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:57:01.999666 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.113.129:60821 (local address 127.15.113.131:46709)
0809 22:57:02.001597 (+ 1931us) server_negotiation.cc:206] Beginning negotiation
0809 22:57:02.001612 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0809 22:57:02.001669 (+ 57us) server_negotiation.cc:407] Connection header received
0809 22:57:02.003825 (+ 2156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:57:02.004177 (+ 352us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:57:02.004576 (+ 399us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:57:02.004865 (+ 289us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:57:02.005969 (+ 1104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:57:02.007155 (+ 1186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:57:02.008695 (+ 1540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:57:02.009357 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:57:02.009514 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:57:02.013058 (+ 3544us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:57:02.013065 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:57:02.013079 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:57:02.013114 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:57:02.016119 (+ 3005us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:57:02.016877 (+ 758us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:57:02.016886 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:57:02.016915 (+ 29us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:57:02.017079 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:57:02.017521 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:57:02.017524 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:57:02.017526 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:57:02.018009 (+ 483us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:57:02.018183 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0809 22:57:02.018948 (+ 765us) server_negotiation.cc:299] Negotiation successful
0809 22:57:02.019288 (+ 340us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1754,"thread_start_us":1791,"threads_started":1}
I20260809 22:57:02.019759 9879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:57:01.998029 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.131:46709 (local address 127.15.113.129:60821)
0809 22:57:01.999717 (+ 1688us) negotiation.cc:107] Waiting for socket to connect
0809 22:57:01.999731 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 22:57:01.999827 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:57:02.004844 (+ 5017us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:57:02.004849 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:57:02.005496 (+ 647us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:57:02.005800 (+ 304us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:57:02.005810 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:57:02.007331 (+ 1521us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:57:02.007337 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:57:02.008552 (+ 1215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:57:02.008565 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:57:02.009583 (+ 1018us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:57:02.009590 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:57:02.009812 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:57:02.010436 (+ 624us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:57:02.010458 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:57:02.012751 (+ 2293us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:57:02.016259 (+ 3508us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:57:02.016267 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:57:02.016279 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:57:02.016753 (+ 474us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:57:02.017198 (+ 445us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:57:02.017202 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:57:02.017204 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:57:02.017389 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:57:02.018151 (+ 762us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:57:02.018159 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:57:02.018426 (+ 267us) client_negotiation.cc:769] Sending connection context
0809 22:57:02.018693 (+ 267us) client_negotiation.cc:240] Negotiation successful
0809 22:57:02.019359 (+ 666us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1591,"mutex_wait_us":289,"spinlock_wait_cycles":768,"thread_start_us":1835,"threads_started":1}
W20260809 22:57:02.023165 9324 leader_election.cc:341] T 692fbd11681248d18ddf09fac28423fd P c988eacb729c44a0bdd072dc42707c58 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer a8594d32d7724c3b831d36b48c70ba60 (127.15.113.131:46709): Not found: Tablet not found: 692fbd11681248d18ddf09fac28423fd
W20260809 22:57:02.023406 9322 leader_election.cc:341] T 692fbd11681248d18ddf09fac28423fd P c988eacb729c44a0bdd072dc42707c58 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 0022f56469854dd5b9783b4e88420ef5 (127.15.113.130:33009): Not found: Tablet not found: 692fbd11681248d18ddf09fac28423fd
I20260809 22:57:02.023537 9322 leader_election.cc:302] T 692fbd11681248d18ddf09fac28423fd P c988eacb729c44a0bdd072dc42707c58 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c988eacb729c44a0bdd072dc42707c58; no voters: 0022f56469854dd5b9783b4e88420ef5, a8594d32d7724c3b831d36b48c70ba60
I20260809 22:57:02.023926 9877 raft_consensus.cc:2726] T 692fbd11681248d18ddf09fac28423fd P c988eacb729c44a0bdd072dc42707c58 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 22:57:02.240638 9885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:57:02.239748 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:39625 (local address 127.15.113.131:53209)
0809 22:57:02.240243 (+ 495us) negotiation.cc:107] Waiting for socket to connect
0809 22:57:02.240374 (+ 131us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.113.190:39625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":370,"thread_start_us":221,"threads_started":1}
I20260809 22:57:02.366158 9879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:57:02.365772 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:39625 (local address 127.15.113.129:46967)
0809 22:57:02.365951 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0809 22:57:02.366026 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.113.190:39625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260809 22:57:02.801604 9886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:57:02.800880 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:39625 (local address 127.15.113.130:51597)
0809 22:57:02.801382 (+ 502us) negotiation.cc:107] Waiting for socket to connect
0809 22:57:02.801461 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.113.190:39625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":405,"thread_start_us":285,"threads_started":1}
I20260809 22:57:03.242139 9887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:57:03.241443 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:39625 (local address 127.15.113.131:34485)
0809 22:57:03.241874 (+ 431us) negotiation.cc:107] Waiting for socket to connect
0809 22:57:03.241954 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.113.190:39625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":336,"thread_start_us":227,"threads_started":1}
I20260809 22:57:03.367537 9888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:57:03.366874 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:39625 (local address 127.15.113.129:56029)
0809 22:57:03.367290 (+ 416us) negotiation.cc:107] Waiting for socket to connect
0809 22:57:03.367377 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.113.190:39625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":321,"thread_start_us":207,"threads_started":1}
I20260809 22:57:03.803040 9889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:57:03.802382 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:39625 (local address 127.15.113.130:47687)
0809 22:57:03.802805 (+ 423us) negotiation.cc:107] Waiting for socket to connect
0809 22:57:03.802880 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.113.190:39625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":329,"thread_start_us":217,"threads_started":1}
I20260809 22:57:04.170027 9891 raft_consensus.cc:482] T 692fbd11681248d18ddf09fac28423fd P c988eacb729c44a0bdd072dc42707c58 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 22:57:04.170215 9891 raft_consensus.cc:504] T 692fbd11681248d18ddf09fac28423fd P c988eacb729c44a0bdd072dc42707c58 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c988eacb729c44a0bdd072dc42707c58" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 40825 } } peers { permanent_uuid: "0022f56469854dd5b9783b4e88420ef5" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 33009 } } peers { permanent_uuid: "a8594d32d7724c3b831d36b48c70ba60" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 46709 } }
I20260809 22:57:04.170858 9891 leader_election.cc:288] T 692fbd11681248d18ddf09fac28423fd P c988eacb729c44a0bdd072dc42707c58 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0022f56469854dd5b9783b4e88420ef5 (127.15.113.130:33009), a8594d32d7724c3b831d36b48c70ba60 (127.15.113.131:46709)
W20260809 22:57:04.171789 9322 leader_election.cc:341] T 692fbd11681248d18ddf09fac28423fd P c988eacb729c44a0bdd072dc42707c58 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 0022f56469854dd5b9783b4e88420ef5 (127.15.113.130:33009): Not found: Tablet not found: 692fbd11681248d18ddf09fac28423fd
W20260809 22:57:04.171952 9324 leader_election.cc:341] T 692fbd11681248d18ddf09fac28423fd P c988eacb729c44a0bdd072dc42707c58 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer a8594d32d7724c3b831d36b48c70ba60 (127.15.113.131:46709): Not found: Tablet not found: 692fbd11681248d18ddf09fac28423fd
I20260809 22:57:04.172067 9324 leader_election.cc:302] T 692fbd11681248d18ddf09fac28423fd P c988eacb729c44a0bdd072dc42707c58 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c988eacb729c44a0bdd072dc42707c58; no voters: 0022f56469854dd5b9783b4e88420ef5, a8594d32d7724c3b831d36b48c70ba60
I20260809 22:57:04.172305 9891 raft_consensus.cc:2726] T 692fbd11681248d18ddf09fac28423fd P c988eacb729c44a0bdd072dc42707c58 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 22:57:04.243644 9892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:57:04.242971 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:39625 (local address 127.15.113.131:39905)
0809 22:57:04.243407 (+ 436us) negotiation.cc:107] Waiting for socket to connect
0809 22:57:04.243488 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.113.190:39625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":332,"thread_start_us":237,"threads_started":1}
I20260809 22:57:04.369071 9894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:57:04.368286 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:39625 (local address 127.15.113.129:47859)
0809 22:57:04.368747 (+ 461us) negotiation.cc:107] Waiting for socket to connect
0809 22:57:04.368848 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.113.190:39625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":362,"thread_start_us":212,"threads_started":1}
I20260809 22:57:04.804519 9895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:57:04.803779 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:39625 (local address 127.15.113.130:33493)
0809 22:57:04.804230 (+ 451us) negotiation.cc:107] Waiting for socket to connect
0809 22:57:04.804322 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.113.190:39625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":320,"thread_start_us":212,"threads_started":1}
I20260809 22:57:05.245198 9897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:57:05.244464 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:39625 (local address 127.15.113.131:57347)
0809 22:57:05.244928 (+ 464us) negotiation.cc:107] Waiting for socket to connect
0809 22:57:05.245012 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.113.190:39625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":337,"thread_start_us":213,"threads_started":1}
I20260809 22:57:05.370621 9898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:57:05.369893 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:39625 (local address 127.15.113.129:50239)
0809 22:57:05.370364 (+ 471us) negotiation.cc:107] Waiting for socket to connect
0809 22:57:05.370452 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.113.190:39625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":383,"thread_start_us":223,"threads_started":1}
I20260809 22:57:05.805994 9899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:57:05.805338 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:39625 (local address 127.15.113.130:56489)
0809 22:57:05.805762 (+ 424us) negotiation.cc:107] Waiting for socket to connect
0809 22:57:05.805838 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.113.190:39625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":333,"thread_start_us":212,"threads_started":1}
I20260809 22:57:06.246716 9902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:57:06.246022 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:39625 (local address 127.15.113.131:58797)
0809 22:57:06.246452 (+ 430us) negotiation.cc:107] Waiting for socket to connect
0809 22:57:06.246540 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.113.190:39625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":337,"thread_start_us":225,"threads_started":1}
I20260809 22:57:06.372087 9904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:57:06.371397 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:39625 (local address 127.15.113.129:52627)
0809 22:57:06.371825 (+ 428us) negotiation.cc:107] Waiting for socket to connect
0809 22:57:06.371904 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.113.190:39625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":336,"thread_start_us":220,"threads_started":1}
I20260809 22:57:06.486562 9905 raft_consensus.cc:482] T 692fbd11681248d18ddf09fac28423fd P c988eacb729c44a0bdd072dc42707c58 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 22:57:06.486739 9905 raft_consensus.cc:504] T 692fbd11681248d18ddf09fac28423fd P c988eacb729c44a0bdd072dc42707c58 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c988eacb729c44a0bdd072dc42707c58" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 40825 } } peers { permanent_uuid: "0022f56469854dd5b9783b4e88420ef5" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 33009 } } peers { permanent_uuid: "a8594d32d7724c3b831d36b48c70ba60" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 46709 } }
I20260809 22:57:06.487423 9905 leader_election.cc:288] T 692fbd11681248d18ddf09fac28423fd P c988eacb729c44a0bdd072dc42707c58 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0022f56469854dd5b9783b4e88420ef5 (127.15.113.130:33009), a8594d32d7724c3b831d36b48c70ba60 (127.15.113.131:46709)
W20260809 22:57:06.488319 9322 leader_election.cc:341] T 692fbd11681248d18ddf09fac28423fd P c988eacb729c44a0bdd072dc42707c58 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 0022f56469854dd5b9783b4e88420ef5 (127.15.113.130:33009): Not found: Tablet not found: 692fbd11681248d18ddf09fac28423fd
W20260809 22:57:06.488497 9324 leader_election.cc:341] T 692fbd11681248d18ddf09fac28423fd P c988eacb729c44a0bdd072dc42707c58 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer a8594d32d7724c3b831d36b48c70ba60 (127.15.113.131:46709): Not found: Tablet not found: 692fbd11681248d18ddf09fac28423fd
I20260809 22:57:06.488636 9324 leader_election.cc:302] T 692fbd11681248d18ddf09fac28423fd P c988eacb729c44a0bdd072dc42707c58 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c988eacb729c44a0bdd072dc42707c58; no voters: 0022f56469854dd5b9783b4e88420ef5, a8594d32d7724c3b831d36b48c70ba60
I20260809 22:57:06.488868 9905 raft_consensus.cc:2726] T 692fbd11681248d18ddf09fac28423fd P c988eacb729c44a0bdd072dc42707c58 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 22:57:06.807461 9906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:57:06.806743 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:39625 (local address 127.15.113.130:44729)
0809 22:57:06.807207 (+ 464us) negotiation.cc:107] Waiting for socket to connect
0809 22:57:06.807286 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.113.190:39625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":360,"thread_start_us":243,"threads_started":1}
I20260809 22:57:07.248209 9907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:57:07.247502 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:39625 (local address 127.15.113.131:60675)
0809 22:57:07.247946 (+ 444us) negotiation.cc:107] Waiting for socket to connect
0809 22:57:07.248025 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.113.190:39625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":351,"thread_start_us":238,"threads_started":1}
I20260809 22:57:07.373533 9908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:57:07.372830 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:39625 (local address 127.15.113.129:55365)
0809 22:57:07.373263 (+ 433us) negotiation.cc:107] Waiting for socket to connect
0809 22:57:07.373352 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.113.190:39625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":333,"thread_start_us":221,"threads_started":1}
I20260809 22:57:07.809094 9909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:57:07.808290 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:39625 (local address 127.15.113.130:47921)
0809 22:57:07.808791 (+ 501us) negotiation.cc:107] Waiting for socket to connect
0809 22:57:07.808879 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.113.190:39625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":397,"thread_start_us":217,"threads_started":1}
I20260809 22:57:08.249917 9911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:57:08.249127 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:39625 (local address 127.15.113.131:48803)
0809 22:57:08.249651 (+ 524us) negotiation.cc:107] Waiting for socket to connect
0809 22:57:08.249738 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.113.190:39625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":416,"thread_start_us":230,"threads_started":1}
I20260809 22:57:08.374912 9912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:57:08.374261 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:39625 (local address 127.15.113.129:49091)
0809 22:57:08.374662 (+ 401us) negotiation.cc:107] Waiting for socket to connect
0809 22:57:08.374742 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.113.190:39625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":303,"thread_start_us":203,"threads_started":1}
I20260809 22:57:08.810563 9913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:57:08.809885 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:39625 (local address 127.15.113.130:43283)
0809 22:57:08.810304 (+ 419us) negotiation.cc:107] Waiting for socket to connect
0809 22:57:08.810388 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.113.190:39625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":319,"thread_start_us":205,"threads_started":1}
I20260809 22:57:08.869032 9914 raft_consensus.cc:482] T 692fbd11681248d18ddf09fac28423fd P c988eacb729c44a0bdd072dc42707c58 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 22:57:08.869328 9914 raft_consensus.cc:504] T 692fbd11681248d18ddf09fac28423fd P c988eacb729c44a0bdd072dc42707c58 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c988eacb729c44a0bdd072dc42707c58" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 40825 } } peers { permanent_uuid: "0022f56469854dd5b9783b4e88420ef5" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 33009 } } peers { permanent_uuid: "a8594d32d7724c3b831d36b48c70ba60" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 46709 } }
I20260809 22:57:08.870034 9914 leader_election.cc:288] T 692fbd11681248d18ddf09fac28423fd P c988eacb729c44a0bdd072dc42707c58 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0022f56469854dd5b9783b4e88420ef5 (127.15.113.130:33009), a8594d32d7724c3b831d36b48c70ba60 (127.15.113.131:46709)
W20260809 22:57:08.870888 9322 leader_election.cc:341] T 692fbd11681248d18ddf09fac28423fd P c988eacb729c44a0bdd072dc42707c58 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 0022f56469854dd5b9783b4e88420ef5 (127.15.113.130:33009): Not found: Tablet not found: 692fbd11681248d18ddf09fac28423fd
W20260809 22:57:08.871064 9324 leader_election.cc:341] T 692fbd11681248d18ddf09fac28423fd P c988eacb729c44a0bdd072dc42707c58 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer a8594d32d7724c3b831d36b48c70ba60 (127.15.113.131:46709): Not found: Tablet not found: 692fbd11681248d18ddf09fac28423fd
I20260809 22:57:08.871188 9324 leader_election.cc:302] T 692fbd11681248d18ddf09fac28423fd P c988eacb729c44a0bdd072dc42707c58 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c988eacb729c44a0bdd072dc42707c58; no voters: 0022f56469854dd5b9783b4e88420ef5, a8594d32d7724c3b831d36b48c70ba60
I20260809 22:57:08.871403 9914 raft_consensus.cc:2726] T 692fbd11681248d18ddf09fac28423fd P c988eacb729c44a0bdd072dc42707c58 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 22:57:09.251353 9915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:57:09.250702 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:39625 (local address 127.15.113.131:38919)
0809 22:57:09.251103 (+ 401us) negotiation.cc:107] Waiting for socket to connect
0809 22:57:09.251180 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.113.190:39625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":306,"thread_start_us":200,"threads_started":1}
I20260809 22:57:09.376292 9916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:57:09.375604 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:39625 (local address 127.15.113.129:51215)
0809 22:57:09.375996 (+ 392us) negotiation.cc:107] Waiting for socket to connect
0809 22:57:09.376083 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.113.190:39625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":301,"thread_start_us":208,"threads_started":1}
I20260809 22:57:09.811997 9917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:57:09.811312 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:39625 (local address 127.15.113.130:52903)
0809 22:57:09.811768 (+ 456us) negotiation.cc:107] Waiting for socket to connect
0809 22:57:09.811857 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.113.190:39625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":354,"thread_start_us":205,"threads_started":1}
I20260809 22:57:10.252929 9918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:57:10.252199 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:39625 (local address 127.15.113.131:46367)
0809 22:57:10.252680 (+ 481us) negotiation.cc:107] Waiting for socket to connect
0809 22:57:10.252759 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.113.190:39625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":365,"thread_start_us":237,"threads_started":1}
I20260809 22:57:10.377869 9920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:57:10.377172 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:39625 (local address 127.15.113.129:58107)
0809 22:57:10.377615 (+ 443us) negotiation.cc:107] Waiting for socket to connect
0809 22:57:10.377706 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.113.190:39625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":340,"thread_start_us":235,"threads_started":1}
I20260809 22:57:10.813472 9923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:57:10.812763 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:39625 (local address 127.15.113.130:48835)
0809 22:57:10.813209 (+ 446us) negotiation.cc:107] Waiting for socket to connect
0809 22:57:10.813292 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.113.190:39625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":344,"thread_start_us":242,"threads_started":1}
I20260809 22:57:11.254396 9924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:57:11.253746 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:39625 (local address 127.15.113.131:32905)
0809 22:57:11.254142 (+ 396us) negotiation.cc:107] Waiting for socket to connect
0809 22:57:11.254226 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.113.190:39625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":306,"thread_start_us":213,"threads_started":1}
I20260809 22:57:11.379307 9925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:57:11.378659 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:39625 (local address 127.15.113.129:49253)
0809 22:57:11.379051 (+ 392us) negotiation.cc:107] Waiting for socket to connect
0809 22:57:11.379129 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.113.190:39625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":304,"thread_start_us":204,"threads_started":1}
I20260809 22:57:11.814976 9927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:57:11.814259 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:39625 (local address 127.15.113.130:49339)
0809 22:57:11.814731 (+ 472us) negotiation.cc:107] Waiting for socket to connect
0809 22:57:11.814813 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.113.190:39625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":363,"thread_start_us":215,"threads_started":1}
I20260809 22:57:12.255889 9930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:57:12.255204 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:39625 (local address 127.15.113.131:60153)
0809 22:57:12.255652 (+ 448us) negotiation.cc:107] Waiting for socket to connect
0809 22:57:12.255738 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.113.190:39625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":347,"thread_start_us":223,"threads_started":1}
I20260809 22:57:12.380800 9933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:57:12.380048 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:39625 (local address 127.15.113.129:36751)
0809 22:57:12.380535 (+ 487us) negotiation.cc:107] Waiting for socket to connect
0809 22:57:12.380639 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.113.190:39625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":391,"thread_start_us":267,"threads_started":1}
I20260809 22:57:12.816439 9935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:57:12.815757 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:39625 (local address 127.15.113.130:51855)
0809 22:57:12.816188 (+ 431us) negotiation.cc:107] Waiting for socket to connect
0809 22:57:12.816265 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.113.190:39625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":336,"thread_start_us":217,"threads_started":1}
I20260809 22:57:13.257397 9936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:57:13.256697 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:39625 (local address 127.15.113.131:49223)
0809 22:57:13.257138 (+ 441us) negotiation.cc:107] Waiting for socket to connect
0809 22:57:13.257218 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.113.190:39625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":350,"thread_start_us":237,"threads_started":1}
I20260809 22:57:13.382270 9937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:57:13.381568 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:39625 (local address 127.15.113.129:33517)
0809 22:57:13.381993 (+ 425us) negotiation.cc:107] Waiting for socket to connect
0809 22:57:13.382087 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.113.190:39625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":330,"thread_start_us":211,"threads_started":1}
I20260809 22:57:13.817953 9940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:57:13.817285 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:39625 (local address 127.15.113.130:40771)
0809 22:57:13.817712 (+ 427us) negotiation.cc:107] Waiting for socket to connect
0809 22:57:13.817801 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.113.190:39625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":318,"thread_start_us":207,"threads_started":1}
I20260809 22:57:14.258924 9941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:57:14.258224 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:39625 (local address 127.15.113.131:60909)
0809 22:57:14.258672 (+ 448us) negotiation.cc:107] Waiting for socket to connect
0809 22:57:14.258768 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.113.190:39625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":343,"thread_start_us":223,"threads_started":1}
I20260809 22:57:14.383800 9943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:57:14.383092 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:39625 (local address 127.15.113.129:37817)
0809 22:57:14.383529 (+ 437us) negotiation.cc:107] Waiting for socket to connect
0809 22:57:14.383610 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.113.190:39625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":341,"thread_start_us":236,"threads_started":1}
I20260809 22:57:14.607108 9944 raft_consensus.cc:482] T 692fbd11681248d18ddf09fac28423fd P c988eacb729c44a0bdd072dc42707c58 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 22:57:14.607339 9944 raft_consensus.cc:504] T 692fbd11681248d18ddf09fac28423fd P c988eacb729c44a0bdd072dc42707c58 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c988eacb729c44a0bdd072dc42707c58" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 40825 } } peers { permanent_uuid: "0022f56469854dd5b9783b4e88420ef5" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 33009 } } peers { permanent_uuid: "a8594d32d7724c3b831d36b48c70ba60" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 46709 } }
I20260809 22:57:14.608018 9944 leader_election.cc:288] T 692fbd11681248d18ddf09fac28423fd P c988eacb729c44a0bdd072dc42707c58 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0022f56469854dd5b9783b4e88420ef5 (127.15.113.130:33009), a8594d32d7724c3b831d36b48c70ba60 (127.15.113.131:46709)
W20260809 22:57:14.608912 9322 leader_election.cc:341] T 692fbd11681248d18ddf09fac28423fd P c988eacb729c44a0bdd072dc42707c58 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 0022f56469854dd5b9783b4e88420ef5 (127.15.113.130:33009): Not found: Tablet not found: 692fbd11681248d18ddf09fac28423fd
W20260809 22:57:14.609079 9324 leader_election.cc:341] T 692fbd11681248d18ddf09fac28423fd P c988eacb729c44a0bdd072dc42707c58 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer a8594d32d7724c3b831d36b48c70ba60 (127.15.113.131:46709): Not found: Tablet not found: 692fbd11681248d18ddf09fac28423fd
I20260809 22:57:14.609198 9324 leader_election.cc:302] T 692fbd11681248d18ddf09fac28423fd P c988eacb729c44a0bdd072dc42707c58 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c988eacb729c44a0bdd072dc42707c58; no voters: 0022f56469854dd5b9783b4e88420ef5, a8594d32d7724c3b831d36b48c70ba60
I20260809 22:57:14.609416 9944 raft_consensus.cc:2726] T 692fbd11681248d18ddf09fac28423fd P c988eacb729c44a0bdd072dc42707c58 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 22:57:14.819458 9945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:57:14.818709 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:39625 (local address 127.15.113.130:39427)
0809 22:57:14.819194 (+ 485us) negotiation.cc:107] Waiting for socket to connect
0809 22:57:14.819274 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.113.190:39625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":378,"thread_start_us":253,"threads_started":1}
I20260809 22:57:15.260465 9947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:57:15.259722 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:39625 (local address 127.15.113.131:56159)
0809 22:57:15.260196 (+ 474us) negotiation.cc:107] Waiting for socket to connect
0809 22:57:15.260276 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.113.190:39625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":370,"thread_start_us":254,"threads_started":1}
I20260809 22:57:15.385572 9948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:57:15.384827 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:39625 (local address 127.15.113.129:57515)
0809 22:57:15.385269 (+ 442us) negotiation.cc:107] Waiting for socket to connect
0809 22:57:15.385348 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.113.190:39625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":350,"thread_start_us":237,"threads_started":1}
I20260809 22:57:15.821197 9949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:57:15.820383 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:39625 (local address 127.15.113.130:36949)
0809 22:57:15.820865 (+ 482us) negotiation.cc:107] Waiting for socket to connect
0809 22:57:15.821022 (+ 157us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.113.190:39625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":377,"thread_start_us":261,"threads_started":1}
I20260809 22:57:16.238956 15814 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskItYPJR/build/asan/bin/kudu pid 9243
I20260809 22:57:16.253422 9950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:57:16.239970 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34220 (local address 127.15.113.129:40825)
0809 22:57:16.240664 (+ 694us) server_negotiation.cc:206] Beginning negotiation
0809 22:57:16.240674 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 22:57:16.240709 (+ 35us) server_negotiation.cc:407] Connection header received
0809 22:57:16.240791 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:57:16.240799 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:57:16.241142 (+ 343us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:57:16.241318 (+ 176us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:57:16.242899 (+ 1581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:57:16.243918 (+ 1019us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:57:16.245232 (+ 1314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:57:16.245788 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:57:16.245883 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:57:16.248288 (+ 2405us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:57:16.248299 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:57:16.248313 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:57:16.248350 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:57:16.251073 (+ 2723us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:57:16.251647 (+ 574us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:57:16.251654 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:57:16.251656 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:57:16.251799 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:57:16.252130 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:57:16.252135 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:57:16.252137 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:57:16.252506 (+ 369us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:57:16.252732 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0809 22:57:16.252953 (+ 221us) server_negotiation.cc:299] Negotiation successful
0809 22:57:16.253165 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":503,"thread_start_us":351,"threads_started":1}
I20260809 22:57:16.254611 9477 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:57:16.262149 9952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:57:16.261459 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:39625 (local address 127.15.113.131:39845)
0809 22:57:16.261918 (+ 459us) negotiation.cc:107] Waiting for socket to connect
0809 22:57:16.262011 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.113.190:39625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":353,"thread_start_us":239,"threads_started":1}
I20260809 22:57:16.340673 15814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskItYPJR/build/asan/bin/kudu with pid 9243
I20260809 22:57:16.364851 15814 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskItYPJR/build/asan/bin/kudu pid 9492
I20260809 22:57:16.377802 9953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:57:16.365588 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40588 (local address 127.15.113.130:33009)
0809 22:57:16.366090 (+ 502us) server_negotiation.cc:206] Beginning negotiation
0809 22:57:16.366094 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 22:57:16.366109 (+ 15us) server_negotiation.cc:407] Connection header received
0809 22:57:16.366172 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:57:16.366176 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:57:16.366234 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:57:16.366327 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:57:16.367624 (+ 1297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:57:16.368421 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:57:16.369787 (+ 1366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:57:16.370230 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:57:16.370335 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:57:16.372982 (+ 2647us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:57:16.372987 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:57:16.372991 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:57:16.373021 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:57:16.375298 (+ 2277us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:57:16.375909 (+ 611us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:57:16.375914 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:57:16.375918 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:57:16.376039 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:57:16.376399 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:57:16.376404 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:57:16.376408 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:57:16.376666 (+ 258us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:57:16.376930 (+ 264us) server_negotiation.cc:1035] Waiting for connection context
0809 22:57:16.377235 (+ 305us) server_negotiation.cc:299] Negotiation successful
0809 22:57:16.377448 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":402,"thread_start_us":249,"threads_started":1}
I20260809 22:57:16.378491 9683 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:57:16.459717 15814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskItYPJR/build/asan/bin/kudu with pid 9492
I20260809 22:57:16.486588 15814 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskItYPJR/build/asan/bin/kudu pid 9692
I20260809 22:57:16.499315 9957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:57:16.487353 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36068 (local address 127.15.113.131:46709)
0809 22:57:16.487931 (+ 578us) server_negotiation.cc:206] Beginning negotiation
0809 22:57:16.487936 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:57:16.487954 (+ 18us) server_negotiation.cc:407] Connection header received
0809 22:57:16.488012 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:57:16.488016 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:57:16.488094 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:57:16.488185 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:57:16.489607 (+ 1422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:57:16.490397 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:57:16.491580 (+ 1183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:57:16.491996 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:57:16.492083 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:57:16.494540 (+ 2457us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:57:16.494545 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:57:16.494549 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:57:16.494580 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:57:16.497239 (+ 2659us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:57:16.497803 (+ 564us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:57:16.497806 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:57:16.497808 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:57:16.497952 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:57:16.498256 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:57:16.498259 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:57:16.498261 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:57:16.498487 (+ 226us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:57:16.498697 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0809 22:57:16.498879 (+ 182us) server_negotiation.cc:299] Negotiation successful
0809 22:57:16.499039 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":451,"thread_start_us":290,"threads_started":1}
I20260809 22:57:16.499980 9863 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:57:16.581812 15814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskItYPJR/build/asan/bin/kudu with pid 9692
2026-08-09T22:57:16Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (64713 ms)
[----------] 30 tests from SecurityITest (300926 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315935617919-15814-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 22:57:16 dist-test-slave-ncp9 krb5kdc[9961](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 22:57:16 dist-test-slave-ncp9 krb5kdc[9961](info): set up 2 sockets
Aug 09 22:57:16 dist-test-slave-ncp9 krb5kdc[9961](info): commencing operation
krb5kdc: starting...
W20260809 22:57:17.826098 15814 mini_kdc.cc:121] Time spent starting KDC: real 1.198s user 0.000s sys 0.042s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315935617919-15814-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315935617919-15814-0/krb5kdc/test-user.keytab.
Aug 09 22:57:18 dist-test-slave-ncp9 krb5kdc[9961](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316238, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T22:57:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T22:57:18Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.113.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.113.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.113.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.190.keytab.
Entry for principal kudu/127.15.113.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.190.keytab.
I20260809 22:57:18.105988 15814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskItYPJR/build/asan/bin/kudu
/tmp/dist-test-taskItYPJR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315935617919-15814-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315935617919-15814-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315935617919-15814-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315935617919-15814-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.113.190:41653
--webserver_interface=127.15.113.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.190.keytab
--principal=kudu/127.15.113.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.113.148:42257
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.113.190:41653
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315935617919-15814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315935617919-15814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315935617919-15814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:57:18.434967 9977 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:57:18.435241 9977 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:57:18.435338 9977 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:57:18.442390 9977 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 22:57:18.442487 9977 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:57:18.442538 9977 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:57:18.442662 9977 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 22:57:18.442729 9977 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 22:57:18.452068 9977 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.113.148:42257
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315935617919-15814-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315935617919-15814-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.113.190:41653
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.190.keytab
--principal=kudu/127.15.113.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.113.190:41653
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315935617919-15814-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.113.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315935617919-15814-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:57:18.453263 9977 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:57:18.454447 9977 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:57:18.465983 9994 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:57:18.466053 9989 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:57:18.466043 9992 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:57:18.466601 9977 server_base.cc:1034] running on GCE node
I20260809 22:57:18.467353 9977 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:57:18.468660 9977 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:57:18.469861 9977 hybrid_clock.cc:648] HybridClock initialized: now 1786316238469813 us; error 32 us; skew 500 ppm
I20260809 22:57:18.470192 9977 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:57:18 dist-test-slave-ncp9 krb5kdc[9961](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316238, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.113.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:57:18.473198 9977 init.cc:377] Logged in from keytab as kudu/127.15.113.190@KRBTEST.COM (short username kudu)
I20260809 22:57:18.474565 9977 webserver.cc:460] Webserver started at http://127.15.113.190:33769/ using document root <none> and password file <none>
I20260809 22:57:18.475214 9977 fs_manager.cc:359] Metadata directory not provided
I20260809 22:57:18.475332 9977 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:57:18.475617 9977 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:57:18.478530 9977 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315935617919-15814-0/minicluster-data/master-0/data/instance:
uuid: "d4c3151ab5d84d0dbcc6a45d281702a8"
format_stamp: "Formatted at 2026-08-09 22:57:18 on dist-test-slave-ncp9"
server_key: "e5965767348695fdbab45497dba5a5ad"
server_key_iv: "a321ab7ed2c50453eea1480328f99efc"
server_key_version: "encryptionkey@0"
I20260809 22:57:18.479269 9977 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315935617919-15814-0/minicluster-data/master-0/wal/instance:
uuid: "d4c3151ab5d84d0dbcc6a45d281702a8"
format_stamp: "Formatted at 2026-08-09 22:57:18 on dist-test-slave-ncp9"
server_key: "e5965767348695fdbab45497dba5a5ad"
server_key_iv: "a321ab7ed2c50453eea1480328f99efc"
server_key_version: "encryptionkey@0"
I20260809 22:57:18.484256 9977 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260809 22:57:18.487625 10005 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:57:18.488823 9977 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260809 22:57:18.489003 9977 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315935617919-15814-0/minicluster-data/master-0/data,/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315935617919-15814-0/minicluster-data/master-0/wal
uuid: "d4c3151ab5d84d0dbcc6a45d281702a8"
format_stamp: "Formatted at 2026-08-09 22:57:18 on dist-test-slave-ncp9"
server_key: "e5965767348695fdbab45497dba5a5ad"
server_key_iv: "a321ab7ed2c50453eea1480328f99efc"
server_key_version: "encryptionkey@0"
I20260809 22:57:18.489184 9977 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315935617919-15814-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315935617919-15814-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315935617919-15814-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:57:18.552264 9977 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:57:18.556620 9977 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:57:18.556926 9977 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:57:18.574265 9977 rpc_server.cc:307] RPC server started. Bound to: 127.15.113.190:41653
I20260809 22:57:18.574286 10075 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.113.190:41653 every 8 connection(s)
I20260809 22:57:18.575572 9977 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315935617919-15814-0/minicluster-data/master-0/data/info.pb
I20260809 22:57:18.579243 10076 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:57:18.581938 15814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskItYPJR/build/asan/bin/kudu as pid 9977
I20260809 22:57:18.582155 15814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315935617919-15814-0/minicluster-data/master-0/wal/instance
I20260809 22:57:18.582993 15814 external_mini_cluster.cc:1428] Setting key cfbc7d4d1eacbfd7909e7ebdf18f8f87
I20260809 22:57:18.591306 10076 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d4c3151ab5d84d0dbcc6a45d281702a8: Bootstrap starting.
I20260809 22:57:18.595017 10076 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d4c3151ab5d84d0dbcc6a45d281702a8: Neither blocks nor log segments found. Creating new log.
I20260809 22:57:18.596271 10076 log.cc:824] T 00000000000000000000000000000000 P d4c3151ab5d84d0dbcc6a45d281702a8: Log is configured to *not* fsync() on all Append() calls
Aug 09 22:57:18 dist-test-slave-ncp9 krb5kdc[9961](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316238, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.113.190@KRBTEST.COM
I20260809 22:57:18.599160 10076 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d4c3151ab5d84d0dbcc6a45d281702a8: No bootstrap required, opened a new log
I20260809 22:57:18.606986 10079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:57:18.585084 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46548 (local address 127.15.113.190:41653)
0809 22:57:18.585794 (+ 710us) server_negotiation.cc:206] Beginning negotiation
0809 22:57:18.585805 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 22:57:18.585844 (+ 39us) server_negotiation.cc:407] Connection header received
0809 22:57:18.587532 (+ 1688us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:57:18.587553 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:57:18.587935 (+ 382us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:57:18.588258 (+ 323us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:57:18.589994 (+ 1736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:57:18.591282 (+ 1288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:57:18.593167 (+ 1885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:57:18.593885 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:57:18.594008 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:57:18.598573 (+ 4565us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:57:18.598585 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:57:18.598604 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:57:18.598654 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:57:18.602658 (+ 4004us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:57:18.603320 (+ 662us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:57:18.603328 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:57:18.603337 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:57:18.603556 (+ 219us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:57:18.603881 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:57:18.603885 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:57:18.603888 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:57:18.604569 (+ 681us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:57:18.604876 (+ 307us) server_negotiation.cc:1035] Waiting for connection context
0809 22:57:18.605214 (+ 338us) server_negotiation.cc:299] Negotiation successful
0809 22:57:18.605639 (+ 425us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":511,"thread_start_us":303,"threads_started":1}
I20260809 22:57:18.609750 10076 raft_consensus.cc:348] T 00000000000000000000000000000000 P d4c3151ab5d84d0dbcc6a45d281702a8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d4c3151ab5d84d0dbcc6a45d281702a8" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 41653 } }
I20260809 22:57:18.610117 10076 raft_consensus.cc:374] T 00000000000000000000000000000000 P d4c3151ab5d84d0dbcc6a45d281702a8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:57:18.610206 10076 raft_consensus.cc:729] T 00000000000000000000000000000000 P d4c3151ab5d84d0dbcc6a45d281702a8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d4c3151ab5d84d0dbcc6a45d281702a8, State: Initialized, Role: FOLLOWER
I20260809 22:57:18.610726 10076 consensus_queue.cc:260] T 00000000000000000000000000000000 P d4c3151ab5d84d0dbcc6a45d281702a8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d4c3151ab5d84d0dbcc6a45d281702a8" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 41653 } }
I20260809 22:57:18.610939 10076 raft_consensus.cc:388] T 00000000000000000000000000000000 P d4c3151ab5d84d0dbcc6a45d281702a8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 22:57:18.611064 10076 raft_consensus.cc:482] T 00000000000000000000000000000000 P d4c3151ab5d84d0dbcc6a45d281702a8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 22:57:18.611218 10076 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d4c3151ab5d84d0dbcc6a45d281702a8 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:57:18.613539 10076 raft_consensus.cc:504] T 00000000000000000000000000000000 P d4c3151ab5d84d0dbcc6a45d281702a8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d4c3151ab5d84d0dbcc6a45d281702a8" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 41653 } }
I20260809 22:57:18.614017 10076 leader_election.cc:302] T 00000000000000000000000000000000 P d4c3151ab5d84d0dbcc6a45d281702a8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d4c3151ab5d84d0dbcc6a45d281702a8; no voters:
I20260809 22:57:18.614552 10076 leader_election.cc:288] T 00000000000000000000000000000000 P d4c3151ab5d84d0dbcc6a45d281702a8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 22:57:18.614665 10083 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d4c3151ab5d84d0dbcc6a45d281702a8 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 22:57:18.615321 10083 raft_consensus.cc:686] T 00000000000000000000000000000000 P d4c3151ab5d84d0dbcc6a45d281702a8 [term 1 LEADER]: Becoming Leader. State: Replica: d4c3151ab5d84d0dbcc6a45d281702a8, State: Running, Role: LEADER
I20260809 22:57:18.615804 10083 consensus_queue.cc:237] T 00000000000000000000000000000000 P d4c3151ab5d84d0dbcc6a45d281702a8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d4c3151ab5d84d0dbcc6a45d281702a8" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 41653 } }
I20260809 22:57:18.616431 10076 sys_catalog.cc:564] T 00000000000000000000000000000000 P d4c3151ab5d84d0dbcc6a45d281702a8 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 22:57:18.618675 10085 sys_catalog.cc:455] T 00000000000000000000000000000000 P d4c3151ab5d84d0dbcc6a45d281702a8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d4c3151ab5d84d0dbcc6a45d281702a8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d4c3151ab5d84d0dbcc6a45d281702a8" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 41653 } } }
I20260809 22:57:18.619056 10085 sys_catalog.cc:458] T 00000000000000000000000000000000 P d4c3151ab5d84d0dbcc6a45d281702a8 [sys.catalog]: This master's current role is: LEADER
I20260809 22:57:18.619175 10087 sys_catalog.cc:455] T 00000000000000000000000000000000 P d4c3151ab5d84d0dbcc6a45d281702a8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d4c3151ab5d84d0dbcc6a45d281702a8. Latest consensus state: current_term: 1 leader_uuid: "d4c3151ab5d84d0dbcc6a45d281702a8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d4c3151ab5d84d0dbcc6a45d281702a8" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 41653 } } }
I20260809 22:57:18.619616 10087 sys_catalog.cc:458] T 00000000000000000000000000000000 P d4c3151ab5d84d0dbcc6a45d281702a8 [sys.catalog]: This master's current role is: LEADER
I20260809 22:57:18.635257 10096 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 22:57:18.640580 10096 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 22:57:18.649205 10096 catalog_manager.cc:1347] Generated new cluster ID: 8edb8409e72b40faafe6b1f8bf0d2d42
I20260809 22:57:18.649307 10096 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 22:57:18.691421 10096 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 22:57:18.692811 10096 catalog_manager.cc:1504] Loading token signing keys...
I20260809 22:57:18.702262 10096 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d4c3151ab5d84d0dbcc6a45d281702a8: Generated new TSK 0
I20260809 22:57:18.703022 10096 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.113.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.113.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.113.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.129.keytab.
Entry for principal kudu/127.15.113.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.129.keytab.
I20260809 22:57:18.776772 15814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskItYPJR/build/asan/bin/kudu
/tmp/dist-test-taskItYPJR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315935617919-15814-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315935617919-15814-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315935617919-15814-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315935617919-15814-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.113.129:0
--local_ip_for_outbound_sockets=127.15.113.129
--webserver_interface=127.15.113.129
--webserver_port=0
--tserver_master_addrs=127.15.113.190:41653
--keytab_file=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.129.keytab
--principal=kudu/127.15.113.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.113.148:42257
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315935617919-15814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315935617919-15814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315935617919-15814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:57:19.116925 10118 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:57:19.117247 10118 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:57:19.117393 10118 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:57:19.124599 10118 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:57:19.124722 10118 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:57:19.124909 10118 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.113.129
I20260809 22:57:19.134305 10118 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.113.148:42257
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315935617919-15814-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315935617919-15814-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.129.keytab
--principal=kudu/127.15.113.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.113.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315935617919-15814-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.113.129
--webserver_port=0
--tserver_master_addrs=127.15.113.190:41653
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.113.129
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315935617919-15814-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:57:19.135632 10118 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:57:19.137017 10118 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:57:19.150158 10126 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:57:19.150403 10125 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:57:19.150195 10129 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:57:19.151084 10118 server_base.cc:1034] running on GCE node
I20260809 22:57:19.151711 10118 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:57:19.152619 10118 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:57:19.153827 10118 hybrid_clock.cc:648] HybridClock initialized: now 1786316239153782 us; error 31 us; skew 500 ppm
I20260809 22:57:19.154147 10118 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:57:19 dist-test-slave-ncp9 krb5kdc[9961](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316239, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.113.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:57:19.160214 10118 init.cc:377] Logged in from keytab as kudu/127.15.113.129@KRBTEST.COM (short username kudu)
I20260809 22:57:19.161738 10118 webserver.cc:460] Webserver started at http://127.15.113.129:41281/ using document root <none> and password file <none>
I20260809 22:57:19.162415 10118 fs_manager.cc:359] Metadata directory not provided
I20260809 22:57:19.162555 10118 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:57:19.162884 10118 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:57:19.165874 10118 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315935617919-15814-0/minicluster-data/ts-0/data/instance:
uuid: "6efc661dae6245a6aee02ddf1b88e3c4"
format_stamp: "Formatted at 2026-08-09 22:57:19 on dist-test-slave-ncp9"
server_key: "f7af56b807784106dc6fc201c5e6bce5"
server_key_iv: "f078ea2be6362b0768d18be50a71e469"
server_key_version: "encryptionkey@0"
I20260809 22:57:19.166679 10118 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315935617919-15814-0/minicluster-data/ts-0/wal/instance:
uuid: "6efc661dae6245a6aee02ddf1b88e3c4"
format_stamp: "Formatted at 2026-08-09 22:57:19 on dist-test-slave-ncp9"
server_key: "f7af56b807784106dc6fc201c5e6bce5"
server_key_iv: "f078ea2be6362b0768d18be50a71e469"
server_key_version: "encryptionkey@0"
I20260809 22:57:19.171833 10118 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260809 22:57:19.175275 10140 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:57:19.176478 10118 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260809 22:57:19.176642 10118 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315935617919-15814-0/minicluster-data/ts-0/data,/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315935617919-15814-0/minicluster-data/ts-0/wal
uuid: "6efc661dae6245a6aee02ddf1b88e3c4"
format_stamp: "Formatted at 2026-08-09 22:57:19 on dist-test-slave-ncp9"
server_key: "f7af56b807784106dc6fc201c5e6bce5"
server_key_iv: "f078ea2be6362b0768d18be50a71e469"
server_key_version: "encryptionkey@0"
I20260809 22:57:19.176828 10118 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315935617919-15814-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315935617919-15814-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315935617919-15814-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:57:19.192732 10118 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:57:19.196580 10118 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:57:19.196859 10118 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:57:19.197831 10118 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:57:19.199155 10118 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:57:19.199235 10118 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:57:19.199334 10118 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:57:19.199389 10118 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:57:19.231621 10118 rpc_server.cc:307] RPC server started. Bound to: 127.15.113.129:42477
I20260809 22:57:19.231662 10306 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.113.129:42477 every 8 connection(s)
I20260809 22:57:19.233003 10118 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315935617919-15814-0/minicluster-data/ts-0/data/info.pb
I20260809 22:57:19.234450 15814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskItYPJR/build/asan/bin/kudu as pid 10118
I20260809 22:57:19.234680 15814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315935617919-15814-0/minicluster-data/ts-0/wal/instance
I20260809 22:57:19.235517 15814 external_mini_cluster.cc:1428] Setting key dd857c922d526b2cf645e82befcc96cf
Aug 09 22:57:19 dist-test-slave-ncp9 krb5kdc[9961](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316239, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.113.129@KRBTEST.COM for kudu/127.15.113.190@KRBTEST.COM
I20260809 22:57:19.252585 10312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:57:19.235781 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.113.129:45995 (local address 127.15.113.190:41653)
0809 22:57:19.236314 (+ 533us) server_negotiation.cc:206] Beginning negotiation
0809 22:57:19.236321 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:57:19.237339 (+ 1018us) server_negotiation.cc:407] Connection header received
0809 22:57:19.239193 (+ 1854us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:57:19.239198 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:57:19.239278 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:57:19.239372 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:57:19.240832 (+ 1460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:57:19.241608 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:57:19.243117 (+ 1509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:57:19.243502 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:57:19.243613 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:57:19.247126 (+ 3513us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:57:19.247131 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:57:19.247134 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:57:19.247168 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:57:19.249563 (+ 2395us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:57:19.250368 (+ 805us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:57:19.250371 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:57:19.250374 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:57:19.250480 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:57:19.250934 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:57:19.250937 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:57:19.250939 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:57:19.251228 (+ 289us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:57:19.251369 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0809 22:57:19.252024 (+ 655us) server_negotiation.cc:299] Negotiation successful
0809 22:57:19.252240 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":377,"thread_start_us":244,"threads_started":1}
I20260809 22:57:19.253414 10313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:57:19.236380 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:41653 (local address 127.15.113.129:45995)
0809 22:57:19.237191 (+ 811us) negotiation.cc:107] Waiting for socket to connect
0809 22:57:19.237230 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0809 22:57:19.238945 (+ 1715us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:57:19.239627 (+ 682us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:57:19.239635 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:57:19.240118 (+ 483us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:57:19.240671 (+ 553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:57:19.240689 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:57:19.241765 (+ 1076us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:57:19.241769 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:57:19.242982 (+ 1213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:57:19.242994 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:57:19.243718 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:57:19.243726 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:57:19.243953 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:57:19.244543 (+ 590us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:57:19.244564 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:57:19.246911 (+ 2347us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:57:19.249697 (+ 2786us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:57:19.249706 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:57:19.249721 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:57:19.250266 (+ 545us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:57:19.250574 (+ 308us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:57:19.250579 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:57:19.250582 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:57:19.250834 (+ 252us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:57:19.251351 (+ 517us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:57:19.251357 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:57:19.251666 (+ 309us) client_negotiation.cc:769] Sending connection context
0809 22:57:19.251908 (+ 242us) client_negotiation.cc:240] Negotiation successful
0809 22:57:19.252277 (+ 369us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":544,"thread_start_us":326,"threads_started":1}
I20260809 22:57:19.254910 10309 heartbeater.cc:347] Connected to a master server at 127.15.113.190:41653
I20260809 22:57:19.255208 10309 heartbeater.cc:464] Registering TS with master...
I20260809 22:57:19.256047 10309 heartbeater.cc:511] Master 127.15.113.190:41653 requested a full tablet report, sending...
I20260809 22:57:19.258172 10026 ts_manager.cc:194] Registered new tserver with Master: 6efc661dae6245a6aee02ddf1b88e3c4 (127.15.113.129:42477)
I20260809 22:57:19.259655 10026 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.113.129@KRBTEST.COM'} at 127.15.113.129:45995
WARNING: no policy specified for kudu/127.15.113.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.113.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.113.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.130.keytab.
Entry for principal kudu/127.15.113.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.130.keytab.
I20260809 22:57:19.302181 15814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskItYPJR/build/asan/bin/kudu
/tmp/dist-test-taskItYPJR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315935617919-15814-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315935617919-15814-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315935617919-15814-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315935617919-15814-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.113.130:0
--local_ip_for_outbound_sockets=127.15.113.130
--webserver_interface=127.15.113.130
--webserver_port=0
--tserver_master_addrs=127.15.113.190:41653
--keytab_file=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.130.keytab
--principal=kudu/127.15.113.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.113.148:42257
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315935617919-15814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315935617919-15814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315935617919-15814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:57:19.628750 10316 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:57:19.629043 10316 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:57:19.629181 10316 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:57:19.636322 10316 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:57:19.636425 10316 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:57:19.636570 10316 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.113.130
I20260809 22:57:19.646096 10316 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.113.148:42257
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315935617919-15814-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315935617919-15814-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.130.keytab
--principal=kudu/127.15.113.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.113.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315935617919-15814-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.113.130
--webserver_port=0
--tserver_master_addrs=127.15.113.190:41653
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.113.130
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315935617919-15814-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:57:19.647316 10316 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:57:19.648633 10316 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:57:19.662416 10322 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:57:19.662513 10327 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:57:19.662453 10323 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:57:19.663300 10316 server_base.cc:1034] running on GCE node
I20260809 22:57:19.663882 10316 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:57:19.664772 10316 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:57:19.665962 10316 hybrid_clock.cc:648] HybridClock initialized: now 1786316239665912 us; error 43 us; skew 500 ppm
I20260809 22:57:19.666270 10316 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:57:19 dist-test-slave-ncp9 krb5kdc[9961](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316239, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.113.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:57:19.669250 10316 init.cc:377] Logged in from keytab as kudu/127.15.113.130@KRBTEST.COM (short username kudu)
I20260809 22:57:19.670642 10316 webserver.cc:460] Webserver started at http://127.15.113.130:33377/ using document root <none> and password file <none>
I20260809 22:57:19.671273 10316 fs_manager.cc:359] Metadata directory not provided
I20260809 22:57:19.671372 10316 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:57:19.671643 10316 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:57:19.674454 10316 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315935617919-15814-0/minicluster-data/ts-1/data/instance:
uuid: "e8c86eae9ed94482b542d4fe1d56b7e2"
format_stamp: "Formatted at 2026-08-09 22:57:19 on dist-test-slave-ncp9"
server_key: "a7c08b83448e891d2031cd9a0d5b7f19"
server_key_iv: "01f69d65eae7f46acd8cd1eb1b6d7caa"
server_key_version: "encryptionkey@0"
I20260809 22:57:19.675171 10316 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315935617919-15814-0/minicluster-data/ts-1/wal/instance:
uuid: "e8c86eae9ed94482b542d4fe1d56b7e2"
format_stamp: "Formatted at 2026-08-09 22:57:19 on dist-test-slave-ncp9"
server_key: "a7c08b83448e891d2031cd9a0d5b7f19"
server_key_iv: "01f69d65eae7f46acd8cd1eb1b6d7caa"
server_key_version: "encryptionkey@0"
I20260809 22:57:19.680208 10316 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260809 22:57:19.683495 10335 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:57:19.684721 10316 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 22:57:19.684859 10316 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315935617919-15814-0/minicluster-data/ts-1/data,/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315935617919-15814-0/minicluster-data/ts-1/wal
uuid: "e8c86eae9ed94482b542d4fe1d56b7e2"
format_stamp: "Formatted at 2026-08-09 22:57:19 on dist-test-slave-ncp9"
server_key: "a7c08b83448e891d2031cd9a0d5b7f19"
server_key_iv: "01f69d65eae7f46acd8cd1eb1b6d7caa"
server_key_version: "encryptionkey@0"
I20260809 22:57:19.685086 10316 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315935617919-15814-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315935617919-15814-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315935617919-15814-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:57:19.710489 10316 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:57:19.714152 10316 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:57:19.714403 10316 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:57:19.715294 10316 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:57:19.716603 10316 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:57:19.716681 10316 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:57:19.716796 10316 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:57:19.716845 10316 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:57:19.746862 10316 rpc_server.cc:307] RPC server started. Bound to: 127.15.113.130:39649
I20260809 22:57:19.746907 10505 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.113.130:39649 every 8 connection(s)
I20260809 22:57:19.748095 10316 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315935617919-15814-0/minicluster-data/ts-1/data/info.pb
I20260809 22:57:19.757915 15814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskItYPJR/build/asan/bin/kudu as pid 10316
I20260809 22:57:19.758091 15814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315935617919-15814-0/minicluster-data/ts-1/wal/instance
I20260809 22:57:19.758524 15814 external_mini_cluster.cc:1428] Setting key 8deaa1a96ea4a3370a1be7b027715533
Aug 09 22:57:19 dist-test-slave-ncp9 krb5kdc[9961](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316239, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.113.130@KRBTEST.COM for kudu/127.15.113.190@KRBTEST.COM
I20260809 22:57:19.767170 10312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:57:19.750560 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.113.130:47901 (local address 127.15.113.190:41653)
0809 22:57:19.750780 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0809 22:57:19.750786 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:57:19.751958 (+ 1172us) server_negotiation.cc:407] Connection header received
0809 22:57:19.753804 (+ 1846us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:57:19.753809 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:57:19.753879 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:57:19.753966 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:57:19.755421 (+ 1455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:57:19.756111 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:57:19.757698 (+ 1587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:57:19.758186 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:57:19.758305 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:57:19.761841 (+ 3536us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:57:19.761846 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:57:19.761849 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:57:19.761882 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:57:19.764256 (+ 2374us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:57:19.764964 (+ 708us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:57:19.764968 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:57:19.764970 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:57:19.765108 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:57:19.765524 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:57:19.765528 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:57:19.765530 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:57:19.765796 (+ 266us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:57:19.765929 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0809 22:57:19.766610 (+ 681us) server_negotiation.cc:299] Negotiation successful
0809 22:57:19.766852 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260809 22:57:19.768079 10508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:57:19.751090 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:41653 (local address 127.15.113.130:47901)
0809 22:57:19.751788 (+ 698us) negotiation.cc:107] Waiting for socket to connect
0809 22:57:19.751838 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0809 22:57:19.753558 (+ 1720us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:57:19.754189 (+ 631us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:57:19.754197 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:57:19.754698 (+ 501us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:57:19.755268 (+ 570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:57:19.755286 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:57:19.756238 (+ 952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:57:19.756241 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:57:19.757527 (+ 1286us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:57:19.757545 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:57:19.758402 (+ 857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:57:19.758410 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:57:19.758649 (+ 239us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:57:19.759262 (+ 613us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:57:19.759284 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:57:19.761622 (+ 2338us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:57:19.764388 (+ 2766us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:57:19.764398 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:57:19.764413 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:57:19.764811 (+ 398us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:57:19.765217 (+ 406us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:57:19.765222 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:57:19.765225 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:57:19.765404 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:57:19.765923 (+ 519us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:57:19.765931 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:57:19.766244 (+ 313us) client_negotiation.cc:769] Sending connection context
0809 22:57:19.766475 (+ 231us) client_negotiation.cc:240] Negotiation successful
0809 22:57:19.766977 (+ 502us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":465,"thread_start_us":267,"threads_started":1}
I20260809 22:57:19.769474 10506 heartbeater.cc:347] Connected to a master server at 127.15.113.190:41653
I20260809 22:57:19.769769 10506 heartbeater.cc:464] Registering TS with master...
I20260809 22:57:19.770444 10506 heartbeater.cc:511] Master 127.15.113.190:41653 requested a full tablet report, sending...
I20260809 22:57:19.771869 10026 ts_manager.cc:194] Registered new tserver with Master: e8c86eae9ed94482b542d4fe1d56b7e2 (127.15.113.130:39649)
I20260809 22:57:19.772859 10026 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.113.130@KRBTEST.COM'} at 127.15.113.130:47901
WARNING: no policy specified for kudu/127.15.113.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.113.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.113.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.131.keytab.
Entry for principal kudu/127.15.113.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.131.keytab.
I20260809 22:57:19.821971 15814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskItYPJR/build/asan/bin/kudu
/tmp/dist-test-taskItYPJR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315935617919-15814-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315935617919-15814-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315935617919-15814-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315935617919-15814-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.113.131:0
--local_ip_for_outbound_sockets=127.15.113.131
--webserver_interface=127.15.113.131
--webserver_port=0
--tserver_master_addrs=127.15.113.190:41653
--keytab_file=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.131.keytab
--principal=kudu/127.15.113.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.113.148:42257
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315935617919-15814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315935617919-15814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315935617919-15814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:57:20.145498 10511 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:57:20.145763 10511 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:57:20.145898 10511 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:57:20.152668 10511 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:57:20.152765 10511 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:57:20.152925 10511 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.113.131
I20260809 22:57:20.161945 10511 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.113.148:42257
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315935617919-15814-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315935617919-15814-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.131.keytab
--principal=kudu/127.15.113.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.113.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315935617919-15814-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.113.131
--webserver_port=0
--tserver_master_addrs=127.15.113.190:41653
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.113.131
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315935617919-15814-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:57:20.163121 10511 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:57:20.164352 10511 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:57:20.176447 10519 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:57:20.176479 10520 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:57:20.176520 10522 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:57:20.177296 10511 server_base.cc:1034] running on GCE node
I20260809 22:57:20.177855 10511 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:57:20.178687 10511 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:57:20.179885 10511 hybrid_clock.cc:648] HybridClock initialized: now 1786316240179843 us; error 32 us; skew 500 ppm
I20260809 22:57:20.180182 10511 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:57:20 dist-test-slave-ncp9 krb5kdc[9961](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316240, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.113.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:57:20.183295 10511 init.cc:377] Logged in from keytab as kudu/127.15.113.131@KRBTEST.COM (short username kudu)
I20260809 22:57:20.184562 10511 webserver.cc:460] Webserver started at http://127.15.113.131:37503/ using document root <none> and password file <none>
I20260809 22:57:20.185241 10511 fs_manager.cc:359] Metadata directory not provided
I20260809 22:57:20.185359 10511 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:57:20.185657 10511 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:57:20.188513 10511 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315935617919-15814-0/minicluster-data/ts-2/data/instance:
uuid: "4b239f3593a84c53a36565628c6e7813"
format_stamp: "Formatted at 2026-08-09 22:57:20 on dist-test-slave-ncp9"
server_key: "adc92212f2bc5a996942cc19745a86fc"
server_key_iv: "c3b9f8be4cfb0508b79196b54c8d9a99"
server_key_version: "encryptionkey@0"
I20260809 22:57:20.189276 10511 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315935617919-15814-0/minicluster-data/ts-2/wal/instance:
uuid: "4b239f3593a84c53a36565628c6e7813"
format_stamp: "Formatted at 2026-08-09 22:57:20 on dist-test-slave-ncp9"
server_key: "adc92212f2bc5a996942cc19745a86fc"
server_key_iv: "c3b9f8be4cfb0508b79196b54c8d9a99"
server_key_version: "encryptionkey@0"
I20260809 22:57:20.194227 10511 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260809 22:57:20.197431 10532 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:57:20.198619 10511 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260809 22:57:20.198766 10511 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315935617919-15814-0/minicluster-data/ts-2/data,/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315935617919-15814-0/minicluster-data/ts-2/wal
uuid: "4b239f3593a84c53a36565628c6e7813"
format_stamp: "Formatted at 2026-08-09 22:57:20 on dist-test-slave-ncp9"
server_key: "adc92212f2bc5a996942cc19745a86fc"
server_key_iv: "c3b9f8be4cfb0508b79196b54c8d9a99"
server_key_version: "encryptionkey@0"
I20260809 22:57:20.198932 10511 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315935617919-15814-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315935617919-15814-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315935617919-15814-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:57:20.217882 10511 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:57:20.221525 10511 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:57:20.221788 10511 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:57:20.222704 10511 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:57:20.223977 10511 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:57:20.224061 10511 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:57:20.224179 10511 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:57:20.224232 10511 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:57:20.253973 10511 rpc_server.cc:307] RPC server started. Bound to: 127.15.113.131:44591
I20260809 22:57:20.254014 10703 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.113.131:44591 every 8 connection(s)
I20260809 22:57:20.255128 10511 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315935617919-15814-0/minicluster-data/ts-2/data/info.pb
I20260809 22:57:20.256654 15814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskItYPJR/build/asan/bin/kudu as pid 10511
I20260809 22:57:20.256835 15814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315935617919-15814-0/minicluster-data/ts-2/wal/instance
I20260809 22:57:20.257506 15814 external_mini_cluster.cc:1428] Setting key 87e30838d89670b34368e6335e70acd6
I20260809 22:57:20.262907 10309 heartbeater.cc:503] Master 127.15.113.190:41653 was elected leader, sending a full tablet report...
Aug 09 22:57:20 dist-test-slave-ncp9 krb5kdc[9961](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316240, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.113.131@KRBTEST.COM for kudu/127.15.113.190@KRBTEST.COM
I20260809 22:57:20.273955 10312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:57:20.257662 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.113.131:53999 (local address 127.15.113.190:41653)
0809 22:57:20.257824 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0809 22:57:20.257828 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 22:57:20.259185 (+ 1357us) server_negotiation.cc:407] Connection header received
0809 22:57:20.260948 (+ 1763us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:57:20.260953 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:57:20.261025 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:57:20.261101 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:57:20.262450 (+ 1349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:57:20.263286 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:57:20.265136 (+ 1850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:57:20.265548 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:57:20.265648 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:57:20.268736 (+ 3088us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:57:20.268741 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:57:20.268744 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:57:20.268775 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:57:20.271058 (+ 2283us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:57:20.271875 (+ 817us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:57:20.271880 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:57:20.271883 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:57:20.272047 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:57:20.272518 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:57:20.272523 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:57:20.272525 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:57:20.272814 (+ 289us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:57:20.272983 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0809 22:57:20.273548 (+ 565us) server_negotiation.cc:299] Negotiation successful
0809 22:57:20.273718 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260809 22:57:20.274765 10707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:57:20.258245 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:41653 (local address 127.15.113.131:53999)
0809 22:57:20.259016 (+ 771us) negotiation.cc:107] Waiting for socket to connect
0809 22:57:20.259060 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 22:57:20.260694 (+ 1634us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:57:20.261294 (+ 600us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:57:20.261303 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:57:20.261778 (+ 475us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:57:20.262326 (+ 548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:57:20.262345 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:57:20.263435 (+ 1090us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:57:20.263441 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:57:20.265017 (+ 1576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:57:20.265030 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:57:20.265712 (+ 682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:57:20.265716 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:57:20.265894 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:57:20.266482 (+ 588us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:57:20.266507 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:57:20.268539 (+ 2032us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:57:20.271183 (+ 2644us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:57:20.271191 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:57:20.271204 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:57:20.271766 (+ 562us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:57:20.272161 (+ 395us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:57:20.272165 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:57:20.272169 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:57:20.272383 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:57:20.272957 (+ 574us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:57:20.272962 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:57:20.273232 (+ 270us) client_negotiation.cc:769] Sending connection context
0809 22:57:20.273466 (+ 234us) client_negotiation.cc:240] Negotiation successful
0809 22:57:20.273801 (+ 335us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":512,"thread_start_us":314,"threads_started":1}
I20260809 22:57:20.275892 10704 heartbeater.cc:347] Connected to a master server at 127.15.113.190:41653
I20260809 22:57:20.276141 10704 heartbeater.cc:464] Registering TS with master...
I20260809 22:57:20.276819 10704 heartbeater.cc:511] Master 127.15.113.190:41653 requested a full tablet report, sending...
I20260809 22:57:20.278152 10026 ts_manager.cc:194] Registered new tserver with Master: 4b239f3593a84c53a36565628c6e7813 (127.15.113.131:44591)
I20260809 22:57:20.278906 10026 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.113.131@KRBTEST.COM'} at 127.15.113.131:53999
I20260809 22:57:20.284106 15814 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 22:57:20.299737 10312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:57:20.288238 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35926 (local address 127.15.113.190:41653)
0809 22:57:20.288385 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0809 22:57:20.288388 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 22:57:20.288944 (+ 556us) server_negotiation.cc:407] Connection header received
0809 22:57:20.289067 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:57:20.289070 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:57:20.289128 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:57:20.289200 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:57:20.290566 (+ 1366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:57:20.291243 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:57:20.292638 (+ 1395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:57:20.293084 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:57:20.293180 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:57:20.295362 (+ 2182us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:57:20.295366 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:57:20.295369 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:57:20.295399 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:57:20.297604 (+ 2205us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:57:20.298190 (+ 586us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:57:20.298193 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:57:20.298195 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:57:20.298310 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:57:20.298588 (+ 278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:57:20.298591 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:57:20.298593 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:57:20.298874 (+ 281us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:57:20.299048 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0809 22:57:20.299305 (+ 257us) server_negotiation.cc:299] Negotiation successful
0809 22:57:20.299451 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260809 22:57:20.303344 10026 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35926:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 22:57:20.309955 10026 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 22:57:20.331213 10723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:57:20.321213 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.130:39649 (local address 127.0.0.1:47682)
0809 22:57:20.322098 (+ 885us) negotiation.cc:107] Waiting for socket to connect
0809 22:57:20.322141 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 22:57:20.322331 (+ 190us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:57:20.323514 (+ 1183us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:57:20.323522 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:57:20.323554 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:57:20.323966 (+ 412us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:57:20.323985 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:57:20.326145 (+ 2160us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:57:20.326150 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:57:20.328384 (+ 2234us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:57:20.328398 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:57:20.330301 (+ 1903us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:57:20.330307 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:57:20.330521 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:57:20.330573 (+ 52us) client_negotiation.cc:769] Sending connection context
0809 22:57:20.330682 (+ 109us) client_negotiation.cc:240] Negotiation successful
0809 22:57:20.330860 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":633,"thread_start_us":900,"threads_started":1}
I20260809 22:57:20.332727 10724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:57:20.321429 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47682 (local address 127.15.113.130:39649)
0809 22:57:20.322880 (+ 1451us) server_negotiation.cc:206] Beginning negotiation
0809 22:57:20.322888 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 22:57:20.322916 (+ 28us) server_negotiation.cc:407] Connection header received
0809 22:57:20.323019 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:57:20.323027 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:57:20.323271 (+ 244us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:57:20.324168 (+ 897us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:57:20.324390 (+ 222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:57:20.325990 (+ 1600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:57:20.328589 (+ 2599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:57:20.330069 (+ 1480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:57:20.332019 (+ 1950us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:57:20.332190 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0809 22:57:20.332260 (+ 70us) server_negotiation.cc:299] Negotiation successful
0809 22:57:20.332421 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1283,"thread_start_us":1176,"threads_started":1}
I20260809 22:57:20.338425 10415 tablet_service.cc:1467] Processing CreateTablet for tablet c4a976bd756d47cba08ce2416fd2d34c (DEFAULT_TABLE table=test-table [id=9c945aab290141b9a027ac7616869fcb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:57:20.339802 10415 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c4a976bd756d47cba08ce2416fd2d34c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:57:20.340804 10723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:57:20.321977 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.131:44591 (local address 127.0.0.1:53216)
0809 22:57:20.331426 (+ 9449us) negotiation.cc:107] Waiting for socket to connect
0809 22:57:20.331444 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 22:57:20.331542 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:57:20.335434 (+ 3892us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:57:20.335439 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:57:20.335463 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:57:20.335669 (+ 206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:57:20.335679 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:57:20.337467 (+ 1788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:57:20.337472 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:57:20.338631 (+ 1159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:57:20.338642 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:57:20.340106 (+ 1464us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:57:20.340112 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:57:20.340283 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:57:20.340314 (+ 31us) client_negotiation.cc:769] Sending connection context
0809 22:57:20.340405 (+ 91us) client_negotiation.cc:240] Negotiation successful
0809 22:57:20.340532 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9363,"thread_start_us":11707,"threads_started":1}
I20260809 22:57:20.341010 10726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:57:20.322014 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53216 (local address 127.15.113.131:44591)
0809 22:57:20.334908 (+ 12894us) server_negotiation.cc:206] Beginning negotiation
0809 22:57:20.334915 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:57:20.334945 (+ 30us) server_negotiation.cc:407] Connection header received
0809 22:57:20.335030 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:57:20.335036 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:57:20.335238 (+ 202us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:57:20.335802 (+ 564us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:57:20.335999 (+ 197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:57:20.337326 (+ 1327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:57:20.338778 (+ 1452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:57:20.339948 (+ 1170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:57:20.340126 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:57:20.340284 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0809 22:57:20.340474 (+ 190us) server_negotiation.cc:299] Negotiation successful
0809 22:57:20.340632 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12729,"thread_start_us":12577,"threads_started":1}
I20260809 22:57:20.341966 10725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:57:20.322871 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.129:42477 (local address 127.0.0.1:51824)
0809 22:57:20.333944 (+ 11073us) negotiation.cc:107] Waiting for socket to connect
0809 22:57:20.333962 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 22:57:20.334090 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:57:20.334847 (+ 757us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:57:20.334852 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:57:20.334877 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:57:20.335181 (+ 304us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:57:20.335193 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:57:20.337164 (+ 1971us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:57:20.337169 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:57:20.339169 (+ 2000us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:57:20.339185 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:57:20.341218 (+ 2033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:57:20.341226 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:57:20.341405 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:57:20.341433 (+ 28us) client_negotiation.cc:769] Sending connection context
0809 22:57:20.341505 (+ 72us) client_negotiation.cc:240] Negotiation successful
0809 22:57:20.341619 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10955,"mutex_wait_us":1,"thread_start_us":11452,"threads_started":1}
I20260809 22:57:20.342021 10728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:57:20.323432 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51824 (local address 127.15.113.129:42477)
0809 22:57:20.333519 (+ 10087us) server_negotiation.cc:206] Beginning negotiation
0809 22:57:20.333528 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 22:57:20.334084 (+ 556us) server_negotiation.cc:407] Connection header received
0809 22:57:20.334228 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:57:20.334236 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:57:20.334549 (+ 313us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:57:20.334747 (+ 198us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:57:20.335353 (+ 606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:57:20.336999 (+ 1646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:57:20.339330 (+ 2331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:57:20.340698 (+ 1368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:57:20.340886 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:57:20.341061 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0809 22:57:20.341568 (+ 507us) server_negotiation.cc:299] Negotiation successful
0809 22:57:20.341732 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9923,"thread_start_us":9785,"threads_started":1}
I20260809 22:57:20.344611 10599 tablet_service.cc:1467] Processing CreateTablet for tablet c4a976bd756d47cba08ce2416fd2d34c (DEFAULT_TABLE table=test-table [id=9c945aab290141b9a027ac7616869fcb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:57:20.345052 10212 tablet_service.cc:1467] Processing CreateTablet for tablet c4a976bd756d47cba08ce2416fd2d34c (DEFAULT_TABLE table=test-table [id=9c945aab290141b9a027ac7616869fcb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:57:20.345813 10599 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c4a976bd756d47cba08ce2416fd2d34c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:57:20.346349 10212 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c4a976bd756d47cba08ce2416fd2d34c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:57:20.355913 10729 tablet_bootstrap.cc:492] T c4a976bd756d47cba08ce2416fd2d34c P 4b239f3593a84c53a36565628c6e7813: Bootstrap starting.
I20260809 22:57:20.359103 10729 tablet_bootstrap.cc:654] T c4a976bd756d47cba08ce2416fd2d34c P 4b239f3593a84c53a36565628c6e7813: Neither blocks nor log segments found. Creating new log.
I20260809 22:57:20.360071 10729 log.cc:824] T c4a976bd756d47cba08ce2416fd2d34c P 4b239f3593a84c53a36565628c6e7813: Log is configured to *not* fsync() on all Append() calls
I20260809 22:57:20.360760 10730 tablet_bootstrap.cc:492] T c4a976bd756d47cba08ce2416fd2d34c P 6efc661dae6245a6aee02ddf1b88e3c4: Bootstrap starting.
I20260809 22:57:20.364463 10730 tablet_bootstrap.cc:654] T c4a976bd756d47cba08ce2416fd2d34c P 6efc661dae6245a6aee02ddf1b88e3c4: Neither blocks nor log segments found. Creating new log.
I20260809 22:57:20.365360 10732 tablet_bootstrap.cc:492] T c4a976bd756d47cba08ce2416fd2d34c P e8c86eae9ed94482b542d4fe1d56b7e2: Bootstrap starting.
I20260809 22:57:20.365727 10730 log.cc:824] T c4a976bd756d47cba08ce2416fd2d34c P 6efc661dae6245a6aee02ddf1b88e3c4: Log is configured to *not* fsync() on all Append() calls
I20260809 22:57:20.367956 10729 tablet_bootstrap.cc:492] T c4a976bd756d47cba08ce2416fd2d34c P 4b239f3593a84c53a36565628c6e7813: No bootstrap required, opened a new log
I20260809 22:57:20.368243 10729 ts_tablet_manager.cc:1397] T c4a976bd756d47cba08ce2416fd2d34c P 4b239f3593a84c53a36565628c6e7813: Time spent bootstrapping tablet: real 0.013s user 0.004s sys 0.008s
I20260809 22:57:20.368742 10730 tablet_bootstrap.cc:492] T c4a976bd756d47cba08ce2416fd2d34c P 6efc661dae6245a6aee02ddf1b88e3c4: No bootstrap required, opened a new log
I20260809 22:57:20.369051 10730 ts_tablet_manager.cc:1397] T c4a976bd756d47cba08ce2416fd2d34c P 6efc661dae6245a6aee02ddf1b88e3c4: Time spent bootstrapping tablet: real 0.009s user 0.008s sys 0.000s
I20260809 22:57:20.369283 10732 tablet_bootstrap.cc:654] T c4a976bd756d47cba08ce2416fd2d34c P e8c86eae9ed94482b542d4fe1d56b7e2: Neither blocks nor log segments found. Creating new log.
I20260809 22:57:20.370553 10732 log.cc:824] T c4a976bd756d47cba08ce2416fd2d34c P e8c86eae9ed94482b542d4fe1d56b7e2: Log is configured to *not* fsync() on all Append() calls
I20260809 22:57:20.374043 10732 tablet_bootstrap.cc:492] T c4a976bd756d47cba08ce2416fd2d34c P e8c86eae9ed94482b542d4fe1d56b7e2: No bootstrap required, opened a new log
I20260809 22:57:20.374312 10732 ts_tablet_manager.cc:1397] T c4a976bd756d47cba08ce2416fd2d34c P e8c86eae9ed94482b542d4fe1d56b7e2: Time spent bootstrapping tablet: real 0.009s user 0.004s sys 0.003s
I20260809 22:57:20.375988 10729 raft_consensus.cc:348] T c4a976bd756d47cba08ce2416fd2d34c P 4b239f3593a84c53a36565628c6e7813 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8c86eae9ed94482b542d4fe1d56b7e2" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 39649 } } peers { permanent_uuid: "6efc661dae6245a6aee02ddf1b88e3c4" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 42477 } } peers { permanent_uuid: "4b239f3593a84c53a36565628c6e7813" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 44591 } }
I20260809 22:57:20.376327 10729 raft_consensus.cc:374] T c4a976bd756d47cba08ce2416fd2d34c P 4b239f3593a84c53a36565628c6e7813 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:57:20.376426 10729 raft_consensus.cc:729] T c4a976bd756d47cba08ce2416fd2d34c P 4b239f3593a84c53a36565628c6e7813 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b239f3593a84c53a36565628c6e7813, State: Initialized, Role: FOLLOWER
I20260809 22:57:20.377020 10729 consensus_queue.cc:260] T c4a976bd756d47cba08ce2416fd2d34c P 4b239f3593a84c53a36565628c6e7813 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8c86eae9ed94482b542d4fe1d56b7e2" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 39649 } } peers { permanent_uuid: "6efc661dae6245a6aee02ddf1b88e3c4" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 42477 } } peers { permanent_uuid: "4b239f3593a84c53a36565628c6e7813" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 44591 } }
I20260809 22:57:20.378101 10704 heartbeater.cc:503] Master 127.15.113.190:41653 was elected leader, sending a full tablet report...
I20260809 22:57:20.378787 10729 ts_tablet_manager.cc:1428] T c4a976bd756d47cba08ce2416fd2d34c P 4b239f3593a84c53a36565628c6e7813: Time spent starting tablet: real 0.010s user 0.010s sys 0.000s
I20260809 22:57:20.379356 10730 raft_consensus.cc:348] T c4a976bd756d47cba08ce2416fd2d34c P 6efc661dae6245a6aee02ddf1b88e3c4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8c86eae9ed94482b542d4fe1d56b7e2" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 39649 } } peers { permanent_uuid: "6efc661dae6245a6aee02ddf1b88e3c4" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 42477 } } peers { permanent_uuid: "4b239f3593a84c53a36565628c6e7813" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 44591 } }
I20260809 22:57:20.379787 10730 raft_consensus.cc:374] T c4a976bd756d47cba08ce2416fd2d34c P 6efc661dae6245a6aee02ddf1b88e3c4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:57:20.379894 10730 raft_consensus.cc:729] T c4a976bd756d47cba08ce2416fd2d34c P 6efc661dae6245a6aee02ddf1b88e3c4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6efc661dae6245a6aee02ddf1b88e3c4, State: Initialized, Role: FOLLOWER
I20260809 22:57:20.380501 10730 consensus_queue.cc:260] T c4a976bd756d47cba08ce2416fd2d34c P 6efc661dae6245a6aee02ddf1b88e3c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8c86eae9ed94482b542d4fe1d56b7e2" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 39649 } } peers { permanent_uuid: "6efc661dae6245a6aee02ddf1b88e3c4" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 42477 } } peers { permanent_uuid: "4b239f3593a84c53a36565628c6e7813" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 44591 } }
I20260809 22:57:20.383955 10730 ts_tablet_manager.cc:1428] T c4a976bd756d47cba08ce2416fd2d34c P 6efc661dae6245a6aee02ddf1b88e3c4: Time spent starting tablet: real 0.015s user 0.014s sys 0.000s
I20260809 22:57:20.385041 10732 raft_consensus.cc:348] T c4a976bd756d47cba08ce2416fd2d34c P e8c86eae9ed94482b542d4fe1d56b7e2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8c86eae9ed94482b542d4fe1d56b7e2" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 39649 } } peers { permanent_uuid: "6efc661dae6245a6aee02ddf1b88e3c4" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 42477 } } peers { permanent_uuid: "4b239f3593a84c53a36565628c6e7813" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 44591 } }
I20260809 22:57:20.385480 10732 raft_consensus.cc:374] T c4a976bd756d47cba08ce2416fd2d34c P e8c86eae9ed94482b542d4fe1d56b7e2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:57:20.385596 10732 raft_consensus.cc:729] T c4a976bd756d47cba08ce2416fd2d34c P e8c86eae9ed94482b542d4fe1d56b7e2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8c86eae9ed94482b542d4fe1d56b7e2, State: Initialized, Role: FOLLOWER
I20260809 22:57:20.386190 10732 consensus_queue.cc:260] T c4a976bd756d47cba08ce2416fd2d34c P e8c86eae9ed94482b542d4fe1d56b7e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8c86eae9ed94482b542d4fe1d56b7e2" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 39649 } } peers { permanent_uuid: "6efc661dae6245a6aee02ddf1b88e3c4" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 42477 } } peers { permanent_uuid: "4b239f3593a84c53a36565628c6e7813" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 44591 } }
I20260809 22:57:20.387239 10506 heartbeater.cc:503] Master 127.15.113.190:41653 was elected leader, sending a full tablet report...
I20260809 22:57:20.387787 10732 ts_tablet_manager.cc:1428] T c4a976bd756d47cba08ce2416fd2d34c P e8c86eae9ed94482b542d4fe1d56b7e2: Time spent starting tablet: real 0.013s user 0.014s sys 0.000s
I20260809 22:57:20.398517 10742 raft_consensus.cc:482] T c4a976bd756d47cba08ce2416fd2d34c P e8c86eae9ed94482b542d4fe1d56b7e2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 22:57:20.398732 10742 raft_consensus.cc:504] T c4a976bd756d47cba08ce2416fd2d34c P e8c86eae9ed94482b542d4fe1d56b7e2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8c86eae9ed94482b542d4fe1d56b7e2" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 39649 } } peers { permanent_uuid: "6efc661dae6245a6aee02ddf1b88e3c4" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 42477 } } peers { permanent_uuid: "4b239f3593a84c53a36565628c6e7813" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 44591 } }
I20260809 22:57:20.400094 10742 leader_election.cc:288] T c4a976bd756d47cba08ce2416fd2d34c P e8c86eae9ed94482b542d4fe1d56b7e2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6efc661dae6245a6aee02ddf1b88e3c4 (127.15.113.129:42477), 4b239f3593a84c53a36565628c6e7813 (127.15.113.131:44591)
I20260809 22:57:20.406798 10728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:57:20.400461 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.113.130:39403 (local address 127.15.113.129:42477)
0809 22:57:20.400665 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0809 22:57:20.400671 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:57:20.401529 (+ 858us) server_negotiation.cc:407] Connection header received
0809 22:57:20.401685 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:57:20.401688 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:57:20.401745 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:57:20.401826 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:57:20.402533 (+ 707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:57:20.403284 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:57:20.405098 (+ 1814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:57:20.405881 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:57:20.406003 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:57:20.406075 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0809 22:57:20.406461 (+ 386us) server_negotiation.cc:299] Negotiation successful
0809 22:57:20.406578 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260809 22:57:20.406800 10726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:57:20.400486 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.113.130:45137 (local address 127.15.113.131:44591)
0809 22:57:20.400695 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0809 22:57:20.400707 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 22:57:20.401468 (+ 761us) server_negotiation.cc:407] Connection header received
0809 22:57:20.401621 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:57:20.401626 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:57:20.401699 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:57:20.401767 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:57:20.402468 (+ 701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:57:20.403358 (+ 890us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:57:20.405172 (+ 1814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:57:20.405917 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:57:20.406063 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:57:20.406131 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0809 22:57:20.406454 (+ 323us) server_negotiation.cc:299] Negotiation successful
0809 22:57:20.406570 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260809 22:57:20.407074 10744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:57:20.400439 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.129:42477 (local address 127.15.113.130:39403)
0809 22:57:20.401416 (+ 977us) negotiation.cc:107] Waiting for socket to connect
0809 22:57:20.401433 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 22:57:20.401560 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:57:20.401854 (+ 294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:57:20.401859 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:57:20.401883 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:57:20.402363 (+ 480us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:57:20.402373 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:57:20.403408 (+ 1035us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:57:20.403414 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:57:20.404950 (+ 1536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:57:20.404967 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:57:20.406103 (+ 1136us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:57:20.406111 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:57:20.406279 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:57:20.406329 (+ 50us) client_negotiation.cc:769] Sending connection context
0809 22:57:20.406408 (+ 79us) client_negotiation.cc:240] Negotiation successful
0809 22:57:20.406526 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":878,"mutex_wait_us":65,"thread_start_us":566,"threads_started":1}
I20260809 22:57:20.407128 10745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:57:20.400359 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.131:44591 (local address 127.15.113.130:45137)
0809 22:57:20.401363 (+ 1004us) negotiation.cc:107] Waiting for socket to connect
0809 22:57:20.401378 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 22:57:20.401493 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:57:20.401804 (+ 311us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:57:20.401809 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:57:20.401832 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:57:20.402333 (+ 501us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:57:20.402347 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:57:20.403489 (+ 1142us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:57:20.403493 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:57:20.405020 (+ 1527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:57:20.405036 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:57:20.406143 (+ 1107us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:57:20.406148 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:57:20.406292 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:57:20.406333 (+ 41us) client_negotiation.cc:769] Sending connection context
0809 22:57:20.406413 (+ 80us) client_negotiation.cc:240] Negotiation successful
0809 22:57:20.406533 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":875,"mutex_wait_us":4,"thread_start_us":689,"threads_started":1}
I20260809 22:57:20.407923 10635 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c4a976bd756d47cba08ce2416fd2d34c" candidate_uuid: "e8c86eae9ed94482b542d4fe1d56b7e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b239f3593a84c53a36565628c6e7813" is_pre_election: true
I20260809 22:57:20.407990 10239 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c4a976bd756d47cba08ce2416fd2d34c" candidate_uuid: "e8c86eae9ed94482b542d4fe1d56b7e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6efc661dae6245a6aee02ddf1b88e3c4" is_pre_election: true
I20260809 22:57:20.408367 10635 raft_consensus.cc:2454] T c4a976bd756d47cba08ce2416fd2d34c P 4b239f3593a84c53a36565628c6e7813 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e8c86eae9ed94482b542d4fe1d56b7e2 in term 0.
I20260809 22:57:20.408411 10239 raft_consensus.cc:2454] T c4a976bd756d47cba08ce2416fd2d34c P 6efc661dae6245a6aee02ddf1b88e3c4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e8c86eae9ed94482b542d4fe1d56b7e2 in term 0.
I20260809 22:57:20.409003 10340 leader_election.cc:302] T c4a976bd756d47cba08ce2416fd2d34c P e8c86eae9ed94482b542d4fe1d56b7e2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4b239f3593a84c53a36565628c6e7813, e8c86eae9ed94482b542d4fe1d56b7e2; no voters:
I20260809 22:57:20.409343 10742 raft_consensus.cc:2781] T c4a976bd756d47cba08ce2416fd2d34c P e8c86eae9ed94482b542d4fe1d56b7e2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 22:57:20.409462 10742 raft_consensus.cc:482] T c4a976bd756d47cba08ce2416fd2d34c P e8c86eae9ed94482b542d4fe1d56b7e2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 22:57:20.409554 10742 raft_consensus.cc:3037] T c4a976bd756d47cba08ce2416fd2d34c P e8c86eae9ed94482b542d4fe1d56b7e2 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:57:20.411674 10742 raft_consensus.cc:504] T c4a976bd756d47cba08ce2416fd2d34c P e8c86eae9ed94482b542d4fe1d56b7e2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8c86eae9ed94482b542d4fe1d56b7e2" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 39649 } } peers { permanent_uuid: "6efc661dae6245a6aee02ddf1b88e3c4" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 42477 } } peers { permanent_uuid: "4b239f3593a84c53a36565628c6e7813" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 44591 } }
I20260809 22:57:20.412269 10742 leader_election.cc:288] T c4a976bd756d47cba08ce2416fd2d34c P e8c86eae9ed94482b542d4fe1d56b7e2 [CANDIDATE]: Term 1 election: Requested vote from peers 6efc661dae6245a6aee02ddf1b88e3c4 (127.15.113.129:42477), 4b239f3593a84c53a36565628c6e7813 (127.15.113.131:44591)
I20260809 22:57:20.412613 10239 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c4a976bd756d47cba08ce2416fd2d34c" candidate_uuid: "e8c86eae9ed94482b542d4fe1d56b7e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6efc661dae6245a6aee02ddf1b88e3c4"
I20260809 22:57:20.412725 10635 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c4a976bd756d47cba08ce2416fd2d34c" candidate_uuid: "e8c86eae9ed94482b542d4fe1d56b7e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b239f3593a84c53a36565628c6e7813"
I20260809 22:57:20.412781 10239 raft_consensus.cc:3037] T c4a976bd756d47cba08ce2416fd2d34c P 6efc661dae6245a6aee02ddf1b88e3c4 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:57:20.412926 10635 raft_consensus.cc:3037] T c4a976bd756d47cba08ce2416fd2d34c P 4b239f3593a84c53a36565628c6e7813 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:57:20.414881 10239 raft_consensus.cc:2454] T c4a976bd756d47cba08ce2416fd2d34c P 6efc661dae6245a6aee02ddf1b88e3c4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e8c86eae9ed94482b542d4fe1d56b7e2 in term 1.
I20260809 22:57:20.415102 10635 raft_consensus.cc:2454] T c4a976bd756d47cba08ce2416fd2d34c P 4b239f3593a84c53a36565628c6e7813 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e8c86eae9ed94482b542d4fe1d56b7e2 in term 1.
I20260809 22:57:20.415220 10339 leader_election.cc:302] T c4a976bd756d47cba08ce2416fd2d34c P e8c86eae9ed94482b542d4fe1d56b7e2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6efc661dae6245a6aee02ddf1b88e3c4, e8c86eae9ed94482b542d4fe1d56b7e2; no voters:
I20260809 22:57:20.415511 10742 raft_consensus.cc:2781] T c4a976bd756d47cba08ce2416fd2d34c P e8c86eae9ed94482b542d4fe1d56b7e2 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 22:57:20.415920 10742 raft_consensus.cc:686] T c4a976bd756d47cba08ce2416fd2d34c P e8c86eae9ed94482b542d4fe1d56b7e2 [term 1 LEADER]: Becoming Leader. State: Replica: e8c86eae9ed94482b542d4fe1d56b7e2, State: Running, Role: LEADER
I20260809 22:57:20.416433 10742 consensus_queue.cc:237] T c4a976bd756d47cba08ce2416fd2d34c P e8c86eae9ed94482b542d4fe1d56b7e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8c86eae9ed94482b542d4fe1d56b7e2" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 39649 } } peers { permanent_uuid: "6efc661dae6245a6aee02ddf1b88e3c4" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 42477 } } peers { permanent_uuid: "4b239f3593a84c53a36565628c6e7813" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 44591 } }
I20260809 22:57:20.421124 10025 catalog_manager.cc:5495] T c4a976bd756d47cba08ce2416fd2d34c P e8c86eae9ed94482b542d4fe1d56b7e2 reported cstate change: term changed from 0 to 1, leader changed from <none> to e8c86eae9ed94482b542d4fe1d56b7e2 (127.15.113.130). New cstate: current_term: 1 leader_uuid: "e8c86eae9ed94482b542d4fe1d56b7e2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8c86eae9ed94482b542d4fe1d56b7e2" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 39649 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6efc661dae6245a6aee02ddf1b88e3c4" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 42477 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4b239f3593a84c53a36565628c6e7813" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 44591 } health_report { overall_health: UNKNOWN } } }
I20260809 22:57:20.451617 10724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:57:20.447001 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47698 (local address 127.15.113.130:39649)
0809 22:57:20.447197 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0809 22:57:20.447202 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:57:20.447219 (+ 17us) server_negotiation.cc:407] Connection header received
0809 22:57:20.447374 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:57:20.447378 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:57:20.447457 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:57:20.447570 (+ 113us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 22:57:20.447861 (+ 291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:57:20.448548 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:57:20.449867 (+ 1319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:57:20.450250 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:57:20.450334 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:57:20.450676 (+ 342us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 22:57:20.451137 (+ 461us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 22:57:20.451238 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0809 22:57:20.451291 (+ 53us) server_negotiation.cc:299] Negotiation successful
0809 22:57:20.451354 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260809 22:57:20.459043 10635 raft_consensus.cc:1261] T c4a976bd756d47cba08ce2416fd2d34c P 4b239f3593a84c53a36565628c6e7813 [term 1 FOLLOWER]: Refusing update from remote peer e8c86eae9ed94482b542d4fe1d56b7e2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 22:57:20.459081 10239 raft_consensus.cc:1261] T c4a976bd756d47cba08ce2416fd2d34c P 6efc661dae6245a6aee02ddf1b88e3c4 [term 1 FOLLOWER]: Refusing update from remote peer e8c86eae9ed94482b542d4fe1d56b7e2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 22:57:20.459969 10746 consensus_queue.cc:1035] T c4a976bd756d47cba08ce2416fd2d34c P e8c86eae9ed94482b542d4fe1d56b7e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6efc661dae6245a6aee02ddf1b88e3c4" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 42477 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 22:57:20.460290 10742 consensus_queue.cc:1035] T c4a976bd756d47cba08ce2416fd2d34c P e8c86eae9ed94482b542d4fe1d56b7e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b239f3593a84c53a36565628c6e7813" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 44591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 22:57:20.474696 10760 mvcc.cc:205] Tried to move back new op lower bound from 7316751320911159296 to 7316751320750092288. Current Snapshot: MvccSnapshot[applied={T|T < 7316751320911159296}]
I20260809 22:57:20.483273 10026 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35926:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 22:57:20.483508 10026 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35926:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 22:57:20.486954 10026 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d4c3151ab5d84d0dbcc6a45d281702a8: Sending DeleteTablet for 3 replicas of tablet c4a976bd756d47cba08ce2416fd2d34c
I20260809 22:57:20.488103 10212 tablet_service.cc:1514] Processing DeleteTablet for tablet c4a976bd756d47cba08ce2416fd2d34c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:57:20 UTC) from {username='kudu', principal='kudu/127.15.113.190@KRBTEST.COM'} at 127.0.0.1:51824
I20260809 22:57:20.488071 10599 tablet_service.cc:1514] Processing DeleteTablet for tablet c4a976bd756d47cba08ce2416fd2d34c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:57:20 UTC) from {username='kudu', principal='kudu/127.15.113.190@KRBTEST.COM'} at 127.0.0.1:53216
I20260809 22:57:20.488297 10415 tablet_service.cc:1514] Processing DeleteTablet for tablet c4a976bd756d47cba08ce2416fd2d34c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:57:20 UTC) from {username='kudu', principal='kudu/127.15.113.190@KRBTEST.COM'} at 127.0.0.1:47682
I20260809 22:57:20.489471 10772 tablet_replica.cc:330] T c4a976bd756d47cba08ce2416fd2d34c P 6efc661dae6245a6aee02ddf1b88e3c4: stopping tablet replica
I20260809 22:57:20.489605 15814 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskItYPJR/build/asan/bin/kudu pid 10118
I20260809 22:57:20.489854 10772 raft_consensus.cc:2229] T c4a976bd756d47cba08ce2416fd2d34c P 6efc661dae6245a6aee02ddf1b88e3c4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 22:57:20.490296 10772 raft_consensus.cc:2258] T c4a976bd756d47cba08ce2416fd2d34c P 6efc661dae6245a6aee02ddf1b88e3c4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 22:57:20.492744 10772 ts_tablet_manager.cc:1905] T c4a976bd756d47cba08ce2416fd2d34c P 6efc661dae6245a6aee02ddf1b88e3c4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 22:57:20.501128 10772 ts_tablet_manager.cc:1918] T c4a976bd756d47cba08ce2416fd2d34c P 6efc661dae6245a6aee02ddf1b88e3c4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 22:57:20.501305 10772 log.cc:1196] T c4a976bd756d47cba08ce2416fd2d34c P 6efc661dae6245a6aee02ddf1b88e3c4: Deleting WAL directory at /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315935617919-15814-0/minicluster-data/ts-0/wal/wals/c4a976bd756d47cba08ce2416fd2d34c
I20260809 22:57:20.501912 10772 ts_tablet_manager.cc:1939] T c4a976bd756d47cba08ce2416fd2d34c P 6efc661dae6245a6aee02ddf1b88e3c4: Deleting consensus metadata
I20260809 22:57:20.504042 10008 catalog_manager.cc:4841] TS 6efc661dae6245a6aee02ddf1b88e3c4 (127.15.113.129:42477): tablet c4a976bd756d47cba08ce2416fd2d34c (table test-table [id=9c945aab290141b9a027ac7616869fcb]) successfully deleted
I20260809 22:57:20.505481 10774 tablet_replica.cc:330] T c4a976bd756d47cba08ce2416fd2d34c P e8c86eae9ed94482b542d4fe1d56b7e2: stopping tablet replica
I20260809 22:57:20.505966 10774 raft_consensus.cc:2229] T c4a976bd756d47cba08ce2416fd2d34c P e8c86eae9ed94482b542d4fe1d56b7e2 [term 1 LEADER]: Raft consensus shutting down.
I20260809 22:57:20.506703 10774 raft_consensus.cc:2258] T c4a976bd756d47cba08ce2416fd2d34c P e8c86eae9ed94482b542d4fe1d56b7e2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 22:57:20.507329 10773 tablet_replica.cc:330] T c4a976bd756d47cba08ce2416fd2d34c P 4b239f3593a84c53a36565628c6e7813: stopping tablet replica
I20260809 22:57:20.508361 10774 ts_tablet_manager.cc:1905] T c4a976bd756d47cba08ce2416fd2d34c P e8c86eae9ed94482b542d4fe1d56b7e2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 22:57:20.509630 10773 raft_consensus.cc:2229] T c4a976bd756d47cba08ce2416fd2d34c P 4b239f3593a84c53a36565628c6e7813 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 22:57:20.510212 10773 raft_consensus.cc:2258] T c4a976bd756d47cba08ce2416fd2d34c P 4b239f3593a84c53a36565628c6e7813 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 22:57:20.512408 10773 ts_tablet_manager.cc:1905] T c4a976bd756d47cba08ce2416fd2d34c P 4b239f3593a84c53a36565628c6e7813: Deleting tablet data with delete state TABLET_DATA_DELETED
Aug 09 22:57:20 dist-test-slave-ncp9 krb5kdc[9961](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316238, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.113.129@KRBTEST.COM
I20260809 22:57:20.515470 10774 ts_tablet_manager.cc:1918] T c4a976bd756d47cba08ce2416fd2d34c P e8c86eae9ed94482b542d4fe1d56b7e2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 22:57:20.515673 10774 log.cc:1196] T c4a976bd756d47cba08ce2416fd2d34c P e8c86eae9ed94482b542d4fe1d56b7e2: Deleting WAL directory at /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315935617919-15814-0/minicluster-data/ts-1/wal/wals/c4a976bd756d47cba08ce2416fd2d34c
I20260809 22:57:20.516256 10774 ts_tablet_manager.cc:1939] T c4a976bd756d47cba08ce2416fd2d34c P e8c86eae9ed94482b542d4fe1d56b7e2: Deleting consensus metadata
I20260809 22:57:20.518066 10008 catalog_manager.cc:4841] TS e8c86eae9ed94482b542d4fe1d56b7e2 (127.15.113.130:39649): tablet c4a976bd756d47cba08ce2416fd2d34c (table test-table [id=9c945aab290141b9a027ac7616869fcb]) successfully deleted
I20260809 22:57:20.519188 10773 ts_tablet_manager.cc:1918] T c4a976bd756d47cba08ce2416fd2d34c P 4b239f3593a84c53a36565628c6e7813: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 22:57:20.519333 10773 log.cc:1196] T c4a976bd756d47cba08ce2416fd2d34c P 4b239f3593a84c53a36565628c6e7813: Deleting WAL directory at /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315935617919-15814-0/minicluster-data/ts-2/wal/wals/c4a976bd756d47cba08ce2416fd2d34c
I20260809 22:57:20.519747 10773 ts_tablet_manager.cc:1939] T c4a976bd756d47cba08ce2416fd2d34c P 4b239f3593a84c53a36565628c6e7813: Deleting consensus metadata
I20260809 22:57:20.521373 10009 catalog_manager.cc:4841] TS 4b239f3593a84c53a36565628c6e7813 (127.15.113.131:44591): tablet c4a976bd756d47cba08ce2416fd2d34c (table test-table [id=9c945aab290141b9a027ac7616869fcb]) successfully deleted
I20260809 22:57:20.522440 10728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:57:20.490818 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51838 (local address 127.15.113.129:42477)
0809 22:57:20.491209 (+ 391us) server_negotiation.cc:206] Beginning negotiation
0809 22:57:20.491217 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 22:57:20.501532 (+ 10315us) server_negotiation.cc:407] Connection header received
0809 22:57:20.501746 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:57:20.501753 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:57:20.501850 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:57:20.501961 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:57:20.504482 (+ 2521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:57:20.505693 (+ 1211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:57:20.508660 (+ 2967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:57:20.509300 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:57:20.509398 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:57:20.515232 (+ 5834us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:57:20.515249 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:57:20.515267 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:57:20.515313 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:57:20.519204 (+ 3891us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:57:20.520178 (+ 974us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:57:20.520188 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:57:20.520191 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:57:20.520350 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:57:20.520812 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:57:20.520817 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:57:20.520820 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:57:20.521386 (+ 566us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:57:20.521729 (+ 343us) server_negotiation.cc:1035] Waiting for connection context
0809 22:57:20.522008 (+ 279us) server_negotiation.cc:299] Negotiation successful
0809 22:57:20.522175 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":245}
I20260809 22:57:20.523048 10301 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:57:20.623679 15814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskItYPJR/build/asan/bin/kudu with pid 10118
I20260809 22:57:20.652421 15814 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskItYPJR/build/asan/bin/kudu pid 10316
Aug 09 22:57:20 dist-test-slave-ncp9 krb5kdc[9961](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316238, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.113.130@KRBTEST.COM
I20260809 22:57:20.668787 10724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:57:20.653255 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47708 (local address 127.15.113.130:39649)
0809 22:57:20.653460 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0809 22:57:20.653466 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:57:20.653490 (+ 24us) server_negotiation.cc:407] Connection header received
0809 22:57:20.653613 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:57:20.653618 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:57:20.653742 (+ 124us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:57:20.653831 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:57:20.655125 (+ 1294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:57:20.655934 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:57:20.657462 (+ 1528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:57:20.657900 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:57:20.658009 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:57:20.662978 (+ 4969us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:57:20.662991 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:57:20.663007 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:57:20.663044 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:57:20.666083 (+ 3039us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:57:20.666734 (+ 651us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:57:20.666742 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:57:20.666744 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:57:20.666893 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:57:20.667264 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:57:20.667268 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:57:20.667270 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:57:20.667652 (+ 382us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:57:20.668004 (+ 352us) server_negotiation.cc:1035] Waiting for connection context
0809 22:57:20.668227 (+ 223us) server_negotiation.cc:299] Negotiation successful
0809 22:57:20.668445 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260809 22:57:20.669502 10497 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:57:20.781314 15814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskItYPJR/build/asan/bin/kudu with pid 10316
I20260809 22:57:20.804651 15814 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskItYPJR/build/asan/bin/kudu pid 10511
Aug 09 22:57:20 dist-test-slave-ncp9 krb5kdc[9961](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316238, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.113.131@KRBTEST.COM
I20260809 22:57:20.818523 10726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:57:20.805422 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53228 (local address 127.15.113.131:44591)
0809 22:57:20.805600 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0809 22:57:20.805605 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:57:20.805622 (+ 17us) server_negotiation.cc:407] Connection header received
0809 22:57:20.805783 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:57:20.805787 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:57:20.805849 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:57:20.805930 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:57:20.807093 (+ 1163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:57:20.807799 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:57:20.809049 (+ 1250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:57:20.809431 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:57:20.809517 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:57:20.813673 (+ 4156us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:57:20.813688 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:57:20.813704 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:57:20.813740 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:57:20.816120 (+ 2380us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:57:20.816769 (+ 649us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:57:20.816779 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:57:20.816783 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:57:20.816963 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:57:20.817286 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:57:20.817289 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:57:20.817291 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:57:20.817647 (+ 356us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:57:20.817893 (+ 246us) server_negotiation.cc:1035] Waiting for connection context
0809 22:57:20.818121 (+ 228us) server_negotiation.cc:299] Negotiation successful
0809 22:57:20.818287 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260809 22:57:20.819100 10697 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:57:20.905592 15814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskItYPJR/build/asan/bin/kudu with pid 10511
I20260809 22:57:20.931105 15814 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskItYPJR/build/asan/bin/kudu pid 9977
I20260809 22:57:20.932232 10070 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:57:21.039897 15814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskItYPJR/build/asan/bin/kudu with pid 9977
2026-08-09T22:57:21Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (4465 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-08-09T22:57:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T22:57:21Z Disabled control of system clock
I20260809 22:57:21.135618 15814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskItYPJR/build/asan/bin/kudu
/tmp/dist-test-taskItYPJR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786315935617919-15814-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786315935617919-15814-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786315935617919-15814-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786315935617919-15814-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.113.190:33897
--webserver_interface=127.15.113.190
--webserver_port=0
--builtin_ntp_servers=127.15.113.148:33743
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.113.190:33897
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 22:57:21.480968 10792 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:57:21.481276 10792 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:57:21.481415 10792 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:57:21.488715 10792 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 22:57:21.488842 10792 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:57:21.488940 10792 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:57:21.489032 10792 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 22:57:21.489099 10792 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 22:57:21.498324 10792 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.113.148:33743
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786315935617919-15814-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786315935617919-15814-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.113.190:33897
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.113.190:33897
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786315935617919-15814-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.113.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786315935617919-15814-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:57:21.499603 10792 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:57:21.501502 10792 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:57:21.513494 10798 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:57:21.513525 10802 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:57:21.513499 10799 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:57:21.514390 10792 server_base.cc:1034] running on GCE node
I20260809 22:57:21.515136 10792 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:57:21.516486 10792 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:57:21.517695 10792 hybrid_clock.cc:648] HybridClock initialized: now 1786316241517650 us; error 33 us; skew 500 ppm
I20260809 22:57:21.518009 10792 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 22:57:21.520099 10792 webserver.cc:460] Webserver started at http://127.15.113.190:36541/ using document root <none> and password file <none>
I20260809 22:57:21.520742 10792 fs_manager.cc:359] Metadata directory not provided
I20260809 22:57:21.520829 10792 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:57:21.521139 10792 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:57:21.523952 10792 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786315935617919-15814-0/minicluster-data/master-0/data/instance:
uuid: "3d79d7fea78343d1b189cbadad6cf10e"
format_stamp: "Formatted at 2026-08-09 22:57:21 on dist-test-slave-ncp9"
server_key: "19c3c3d21c99b3c54417b839ebc2acac"
server_key_iv: "70839789afd7a7c595ce346a458e5cfc"
server_key_version: "encryptionkey@0"
I20260809 22:57:21.524698 10792 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786315935617919-15814-0/minicluster-data/master-0/wal/instance:
uuid: "3d79d7fea78343d1b189cbadad6cf10e"
format_stamp: "Formatted at 2026-08-09 22:57:21 on dist-test-slave-ncp9"
server_key: "19c3c3d21c99b3c54417b839ebc2acac"
server_key_iv: "70839789afd7a7c595ce346a458e5cfc"
server_key_version: "encryptionkey@0"
I20260809 22:57:21.529503 10792 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260809 22:57:21.532698 10812 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:57:21.533944 10792 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 22:57:21.534087 10792 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786315935617919-15814-0/minicluster-data/master-0/data,/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786315935617919-15814-0/minicluster-data/master-0/wal
uuid: "3d79d7fea78343d1b189cbadad6cf10e"
format_stamp: "Formatted at 2026-08-09 22:57:21 on dist-test-slave-ncp9"
server_key: "19c3c3d21c99b3c54417b839ebc2acac"
server_key_iv: "70839789afd7a7c595ce346a458e5cfc"
server_key_version: "encryptionkey@0"
I20260809 22:57:21.534255 10792 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786315935617919-15814-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786315935617919-15814-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786315935617919-15814-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:57:21.560218 10792 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:57:21.561014 10792 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:57:21.561254 10792 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:57:21.578007 10792 rpc_server.cc:307] RPC server started. Bound to: 127.15.113.190:33897
I20260809 22:57:21.578027 10881 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.113.190:33897 every 8 connection(s)
I20260809 22:57:21.579257 10792 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786315935617919-15814-0/minicluster-data/master-0/data/info.pb
I20260809 22:57:21.580176 15814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskItYPJR/build/asan/bin/kudu as pid 10792
I20260809 22:57:21.580365 15814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786315935617919-15814-0/minicluster-data/master-0/wal/instance
I20260809 22:57:21.580951 15814 external_mini_cluster.cc:1428] Setting key 33e9e9f836b399ef6e3d9213c1e88686
I20260809 22:57:21.583621 10882 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:57:21.594306 10886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:57:21.582857 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35882 (local address 127.15.113.190:33897)
0809 22:57:21.583794 (+ 937us) server_negotiation.cc:206] Beginning negotiation
0809 22:57:21.583805 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 22:57:21.583848 (+ 43us) server_negotiation.cc:407] Connection header received
0809 22:57:21.585488 (+ 1640us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:57:21.585510 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:57:21.585970 (+ 460us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:57:21.586288 (+ 318us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:57:21.586881 (+ 593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:57:21.588179 (+ 1298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:57:21.590064 (+ 1885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:57:21.590733 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:57:21.590860 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:57:21.592084 (+ 1224us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:57:21.592093 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:57:21.592106 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 22:57:21.592647 (+ 541us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:57:21.592720 (+ 73us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 22:57:21.592744 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:57:21.592834 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0809 22:57:21.593081 (+ 247us) server_negotiation.cc:299] Negotiation successful
0809 22:57:21.593415 (+ 334us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":522,"thread_start_us":333,"threads_started":1}
I20260809 22:57:21.595562 10882 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3d79d7fea78343d1b189cbadad6cf10e: Bootstrap starting.
I20260809 22:57:21.598601 10882 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3d79d7fea78343d1b189cbadad6cf10e: Neither blocks nor log segments found. Creating new log.
I20260809 22:57:21.599531 10882 log.cc:824] T 00000000000000000000000000000000 P 3d79d7fea78343d1b189cbadad6cf10e: Log is configured to *not* fsync() on all Append() calls
I20260809 22:57:21.602084 10882 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3d79d7fea78343d1b189cbadad6cf10e: No bootstrap required, opened a new log
I20260809 22:57:21.608958 10882 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3d79d7fea78343d1b189cbadad6cf10e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d79d7fea78343d1b189cbadad6cf10e" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 33897 } }
I20260809 22:57:21.609282 10882 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3d79d7fea78343d1b189cbadad6cf10e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:57:21.609366 10882 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3d79d7fea78343d1b189cbadad6cf10e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d79d7fea78343d1b189cbadad6cf10e, State: Initialized, Role: FOLLOWER
I20260809 22:57:21.609963 10882 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3d79d7fea78343d1b189cbadad6cf10e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d79d7fea78343d1b189cbadad6cf10e" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 33897 } }
I20260809 22:57:21.610211 10882 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3d79d7fea78343d1b189cbadad6cf10e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 22:57:21.610347 10882 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3d79d7fea78343d1b189cbadad6cf10e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 22:57:21.610508 10882 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3d79d7fea78343d1b189cbadad6cf10e [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:57:21.612524 10882 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3d79d7fea78343d1b189cbadad6cf10e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d79d7fea78343d1b189cbadad6cf10e" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 33897 } }
I20260809 22:57:21.613047 10882 leader_election.cc:302] T 00000000000000000000000000000000 P 3d79d7fea78343d1b189cbadad6cf10e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3d79d7fea78343d1b189cbadad6cf10e; no voters:
I20260809 22:57:21.613591 10882 leader_election.cc:288] T 00000000000000000000000000000000 P 3d79d7fea78343d1b189cbadad6cf10e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 22:57:21.613725 10888 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3d79d7fea78343d1b189cbadad6cf10e [term 1 FOLLOWER]: Leader election won for term 1
I20260809 22:57:21.614374 10888 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3d79d7fea78343d1b189cbadad6cf10e [term 1 LEADER]: Becoming Leader. State: Replica: 3d79d7fea78343d1b189cbadad6cf10e, State: Running, Role: LEADER
I20260809 22:57:21.614866 10888 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3d79d7fea78343d1b189cbadad6cf10e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d79d7fea78343d1b189cbadad6cf10e" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 33897 } }
I20260809 22:57:21.615456 10882 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3d79d7fea78343d1b189cbadad6cf10e [sys.catalog]: configured and running, proceeding with master startup.
I20260809 22:57:21.617636 10890 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3d79d7fea78343d1b189cbadad6cf10e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3d79d7fea78343d1b189cbadad6cf10e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d79d7fea78343d1b189cbadad6cf10e" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 33897 } } }
I20260809 22:57:21.618544 10890 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3d79d7fea78343d1b189cbadad6cf10e [sys.catalog]: This master's current role is: LEADER
I20260809 22:57:21.618191 10891 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3d79d7fea78343d1b189cbadad6cf10e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3d79d7fea78343d1b189cbadad6cf10e. Latest consensus state: current_term: 1 leader_uuid: "3d79d7fea78343d1b189cbadad6cf10e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d79d7fea78343d1b189cbadad6cf10e" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 33897 } } }
I20260809 22:57:21.618984 10891 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3d79d7fea78343d1b189cbadad6cf10e [sys.catalog]: This master's current role is: LEADER
I20260809 22:57:21.620051 10903 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 22:57:21.624431 10903 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 22:57:21.631947 10903 catalog_manager.cc:1347] Generated new cluster ID: 18286ff377b54e598ea6e1b3cb3ca2ce
I20260809 22:57:21.632050 10903 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 22:57:21.640616 10903 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 22:57:21.642282 10903 catalog_manager.cc:1504] Loading token signing keys...
I20260809 22:57:21.652912 10903 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3d79d7fea78343d1b189cbadad6cf10e: Generated new TSK 0
I20260809 22:57:21.653867 10903 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 22:57:21.667459 15814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskItYPJR/build/asan/bin/kudu
/tmp/dist-test-taskItYPJR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786315935617919-15814-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786315935617919-15814-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786315935617919-15814-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786315935617919-15814-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.113.129:0
--local_ip_for_outbound_sockets=127.15.113.129
--webserver_interface=127.15.113.129
--webserver_port=0
--tserver_master_addrs=127.15.113.190:33897
--builtin_ntp_servers=127.15.113.148:33743
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 22:57:21.999887 10920 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:57:22.000154 10920 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:57:22.000278 10920 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:57:22.007302 10920 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:57:22.007407 10920 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:57:22.007565 10920 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.113.129
I20260809 22:57:22.016443 10920 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.113.148:33743
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786315935617919-15814-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786315935617919-15814-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.113.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786315935617919-15814-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.113.129
--webserver_port=0
--tserver_master_addrs=127.15.113.190:33897
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.113.129
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786315935617919-15814-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:57:22.017758 10920 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:57:22.019518 10920 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:57:22.031898 10929 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:57:22.031857 10930 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:57:22.032182 10934 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:57:22.032819 10920 server_base.cc:1034] running on GCE node
I20260809 22:57:22.033414 10920 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:57:22.034276 10920 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:57:22.035460 10920 hybrid_clock.cc:648] HybridClock initialized: now 1786316242035407 us; error 45 us; skew 500 ppm
I20260809 22:57:22.035738 10920 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 22:57:22.037917 10920 webserver.cc:460] Webserver started at http://127.15.113.129:43535/ using document root <none> and password file <none>
I20260809 22:57:22.038616 10920 fs_manager.cc:359] Metadata directory not provided
I20260809 22:57:22.038710 10920 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:57:22.039006 10920 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:57:22.042940 10920 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786315935617919-15814-0/minicluster-data/ts-0/data/instance:
uuid: "135dbe5f19b04fa3a83a683b52dee272"
format_stamp: "Formatted at 2026-08-09 22:57:22 on dist-test-slave-ncp9"
server_key: "a3cb6e70a6270b20193bf8bf441f17c3"
server_key_iv: "a69a1c56509029b7413392e52ce97d4a"
server_key_version: "encryptionkey@0"
I20260809 22:57:22.043959 10920 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786315935617919-15814-0/minicluster-data/ts-0/wal/instance:
uuid: "135dbe5f19b04fa3a83a683b52dee272"
format_stamp: "Formatted at 2026-08-09 22:57:22 on dist-test-slave-ncp9"
server_key: "a3cb6e70a6270b20193bf8bf441f17c3"
server_key_iv: "a69a1c56509029b7413392e52ce97d4a"
server_key_version: "encryptionkey@0"
I20260809 22:57:22.050570 10920 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.008s sys 0.000s
I20260809 22:57:22.053821 10941 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:57:22.055042 10920 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260809 22:57:22.055186 10920 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786315935617919-15814-0/minicluster-data/ts-0/data,/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786315935617919-15814-0/minicluster-data/ts-0/wal
uuid: "135dbe5f19b04fa3a83a683b52dee272"
format_stamp: "Formatted at 2026-08-09 22:57:22 on dist-test-slave-ncp9"
server_key: "a3cb6e70a6270b20193bf8bf441f17c3"
server_key_iv: "a69a1c56509029b7413392e52ce97d4a"
server_key_version: "encryptionkey@0"
I20260809 22:57:22.055379 10920 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786315935617919-15814-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786315935617919-15814-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786315935617919-15814-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:57:22.078869 10920 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:57:22.079629 10920 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:57:22.079902 10920 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:57:22.080797 10920 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:57:22.082234 10920 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:57:22.082324 10920 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:57:22.082424 10920 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:57:22.082510 10920 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:57:22.112226 10920 rpc_server.cc:307] RPC server started. Bound to: 127.15.113.129:37199
I20260809 22:57:22.112303 11128 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.113.129:37199 every 8 connection(s)
I20260809 22:57:22.113516 10920 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786315935617919-15814-0/minicluster-data/ts-0/data/info.pb
I20260809 22:57:22.119547 15814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskItYPJR/build/asan/bin/kudu as pid 10920
I20260809 22:57:22.119717 15814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786315935617919-15814-0/minicluster-data/ts-0/wal/instance
I20260809 22:57:22.120292 15814 external_mini_cluster.cc:1428] Setting key 89e1445a8c0d210a3311d2956e353de9
I20260809 22:57:22.123304 15814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskItYPJR/build/asan/bin/kudu
/tmp/dist-test-taskItYPJR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786315935617919-15814-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786315935617919-15814-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786315935617919-15814-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786315935617919-15814-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.113.130:0
--local_ip_for_outbound_sockets=127.15.113.130
--webserver_interface=127.15.113.130
--webserver_port=0
--tserver_master_addrs=127.15.113.190:33897
--builtin_ntp_servers=127.15.113.148:33743
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260809 22:57:22.126978 11133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:57:22.116040 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.113.129:35759 (local address 127.15.113.190:33897)
0809 22:57:22.116549 (+ 509us) server_negotiation.cc:206] Beginning negotiation
0809 22:57:22.116554 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:57:22.117724 (+ 1170us) server_negotiation.cc:407] Connection header received
0809 22:57:22.119916 (+ 2192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:57:22.119923 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:57:22.120009 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:57:22.120155 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:57:22.121279 (+ 1124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:57:22.122323 (+ 1044us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:57:22.124408 (+ 2085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:57:22.124805 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:57:22.124933 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:57:22.126308 (+ 1375us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:57:22.126313 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:57:22.126316 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 22:57:22.126332 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:57:22.126370 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 22:57:22.126381 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:57:22.126444 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0809 22:57:22.126660 (+ 216us) server_negotiation.cc:299] Negotiation successful
0809 22:57:22.126731 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":366,"thread_start_us":262,"threads_started":1}
I20260809 22:57:22.128132 11134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:57:22.116672 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:33897 (local address 127.15.113.129:35759)
0809 22:57:22.117569 (+ 897us) negotiation.cc:107] Waiting for socket to connect
0809 22:57:22.117608 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0809 22:57:22.119247 (+ 1639us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:57:22.120317 (+ 1070us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:57:22.120324 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:57:22.120424 (+ 100us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:57:22.121088 (+ 664us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:57:22.121114 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:57:22.122470 (+ 1356us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:57:22.122475 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:57:22.124277 (+ 1802us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:57:22.124293 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:57:22.125042 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:57:22.125049 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:57:22.125308 (+ 259us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:57:22.125976 (+ 668us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 22:57:22.125980 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:57:22.126029 (+ 49us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 22:57:22.126121 (+ 92us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:57:22.126461 (+ 340us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:57:22.126471 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:57:22.126479 (+ 8us) client_negotiation.cc:769] Sending connection context
0809 22:57:22.126636 (+ 157us) client_negotiation.cc:240] Negotiation successful
0809 22:57:22.126993 (+ 357us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":471,"thread_start_us":275,"threads_started":1}
I20260809 22:57:22.129482 11130 heartbeater.cc:347] Connected to a master server at 127.15.113.190:33897
I20260809 22:57:22.129763 11130 heartbeater.cc:464] Registering TS with master...
I20260809 22:57:22.130456 11130 heartbeater.cc:511] Master 127.15.113.190:33897 requested a full tablet report, sending...
I20260809 22:57:22.132259 10835 ts_manager.cc:194] Registered new tserver with Master: 135dbe5f19b04fa3a83a683b52dee272 (127.15.113.129:37199)
I20260809 22:57:22.133930 10835 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.15.113.129:35759
W20260809 22:57:22.449218 11136 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:57:22.449468 11136 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:57:22.449558 11136 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:57:22.456368 11136 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:57:22.456449 11136 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:57:22.456552 11136 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.113.130
I20260809 22:57:22.465103 11136 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.113.148:33743
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786315935617919-15814-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786315935617919-15814-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.113.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786315935617919-15814-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.113.130
--webserver_port=0
--tserver_master_addrs=127.15.113.190:33897
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.113.130
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786315935617919-15814-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:57:22.466183 11136 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:57:22.467815 11136 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:57:22.480064 11143 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:57:22.480149 11144 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:57:22.480104 11148 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:57:22.481112 11136 server_base.cc:1034] running on GCE node
I20260809 22:57:22.481683 11136 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:57:22.482554 11136 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:57:22.483753 11136 hybrid_clock.cc:648] HybridClock initialized: now 1786316242483704 us; error 44 us; skew 500 ppm
I20260809 22:57:22.484040 11136 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 22:57:22.486068 11136 webserver.cc:460] Webserver started at http://127.15.113.130:42695/ using document root <none> and password file <none>
I20260809 22:57:22.486708 11136 fs_manager.cc:359] Metadata directory not provided
I20260809 22:57:22.486806 11136 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:57:22.487087 11136 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:57:22.489825 11136 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786315935617919-15814-0/minicluster-data/ts-1/data/instance:
uuid: "c5c76dc3b3ac445885bf42f67890c49b"
format_stamp: "Formatted at 2026-08-09 22:57:22 on dist-test-slave-ncp9"
server_key: "53f9c57a284eb2ff6eabddcbab303297"
server_key_iv: "199b99682dc572bf43db94d27ec8c575"
server_key_version: "encryptionkey@0"
I20260809 22:57:22.490573 11136 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786315935617919-15814-0/minicluster-data/ts-1/wal/instance:
uuid: "c5c76dc3b3ac445885bf42f67890c49b"
format_stamp: "Formatted at 2026-08-09 22:57:22 on dist-test-slave-ncp9"
server_key: "53f9c57a284eb2ff6eabddcbab303297"
server_key_iv: "199b99682dc572bf43db94d27ec8c575"
server_key_version: "encryptionkey@0"
I20260809 22:57:22.495410 11136 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260809 22:57:22.498600 11159 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:57:22.499797 11136 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260809 22:57:22.499946 11136 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786315935617919-15814-0/minicluster-data/ts-1/data,/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786315935617919-15814-0/minicluster-data/ts-1/wal
uuid: "c5c76dc3b3ac445885bf42f67890c49b"
format_stamp: "Formatted at 2026-08-09 22:57:22 on dist-test-slave-ncp9"
server_key: "53f9c57a284eb2ff6eabddcbab303297"
server_key_iv: "199b99682dc572bf43db94d27ec8c575"
server_key_version: "encryptionkey@0"
I20260809 22:57:22.500133 11136 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786315935617919-15814-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786315935617919-15814-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786315935617919-15814-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:57:22.515100 11136 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:57:22.515872 11136 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:57:22.516185 11136 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:57:22.517166 11136 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:57:22.518565 11136 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:57:22.518659 11136 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:57:22.518767 11136 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:57:22.518836 11136 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:57:22.547474 11136 rpc_server.cc:307] RPC server started. Bound to: 127.15.113.130:37447
I20260809 22:57:22.547514 11320 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.113.130:37447 every 8 connection(s)
I20260809 22:57:22.548722 11136 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786315935617919-15814-0/minicluster-data/ts-1/data/info.pb
I20260809 22:57:22.551936 15814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskItYPJR/build/asan/bin/kudu as pid 11136
I20260809 22:57:22.552130 15814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786315935617919-15814-0/minicluster-data/ts-1/wal/instance
I20260809 22:57:22.552726 15814 external_mini_cluster.cc:1428] Setting key 79d3ef50026498d54481f7e1811a18bd
I20260809 22:57:22.555877 15814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskItYPJR/build/asan/bin/kudu
/tmp/dist-test-taskItYPJR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786315935617919-15814-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786315935617919-15814-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786315935617919-15814-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786315935617919-15814-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.113.131:0
--local_ip_for_outbound_sockets=127.15.113.131
--webserver_interface=127.15.113.131
--webserver_port=0
--tserver_master_addrs=127.15.113.190:33897
--builtin_ntp_servers=127.15.113.148:33743
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260809 22:57:22.561985 11133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:57:22.551176 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.113.130:51941 (local address 127.15.113.190:33897)
0809 22:57:22.551345 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0809 22:57:22.551349 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 22:57:22.552953 (+ 1604us) server_negotiation.cc:407] Connection header received
0809 22:57:22.555138 (+ 2185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:57:22.555142 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:57:22.555209 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:57:22.555284 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:57:22.556620 (+ 1336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:57:22.557361 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:57:22.559440 (+ 2079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:57:22.559866 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:57:22.559958 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:57:22.561362 (+ 1404us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:57:22.561367 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:57:22.561370 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 22:57:22.561386 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:57:22.561416 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 22:57:22.561427 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:57:22.561495 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0809 22:57:22.561696 (+ 201us) server_negotiation.cc:299] Negotiation successful
0809 22:57:22.561769 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260809 22:57:22.563253 11323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:57:22.551698 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:33897 (local address 127.15.113.130:51941)
0809 22:57:22.552760 (+ 1062us) negotiation.cc:107] Waiting for socket to connect
0809 22:57:22.552818 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0809 22:57:22.554854 (+ 2036us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:57:22.555551 (+ 697us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:57:22.555562 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:57:22.555678 (+ 116us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:57:22.556463 (+ 785us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:57:22.556486 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:57:22.557532 (+ 1046us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:57:22.557538 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:57:22.559274 (+ 1736us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:57:22.559289 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:57:22.560070 (+ 781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:57:22.560084 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:57:22.560334 (+ 250us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:57:22.561034 (+ 700us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 22:57:22.561037 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:57:22.561094 (+ 57us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 22:57:22.561185 (+ 91us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:57:22.561510 (+ 325us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:57:22.561519 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:57:22.561526 (+ 7us) client_negotiation.cc:769] Sending connection context
0809 22:57:22.561689 (+ 163us) client_negotiation.cc:240] Negotiation successful
0809 22:57:22.562040 (+ 351us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":501,"thread_start_us":304,"threads_started":1}
I20260809 22:57:22.564477 11321 heartbeater.cc:347] Connected to a master server at 127.15.113.190:33897
I20260809 22:57:22.564769 11321 heartbeater.cc:464] Registering TS with master...
I20260809 22:57:22.565620 11321 heartbeater.cc:511] Master 127.15.113.190:33897 requested a full tablet report, sending...
I20260809 22:57:22.566984 10835 ts_manager.cc:194] Registered new tserver with Master: c5c76dc3b3ac445885bf42f67890c49b (127.15.113.130:37447)
I20260809 22:57:22.567780 10835 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.15.113.130:51941
W20260809 22:57:22.891484 11324 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:57:22.891839 11324 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:57:22.891990 11324 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:57:22.899187 11324 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:57:22.899284 11324 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:57:22.899462 11324 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.113.131
I20260809 22:57:22.908454 11324 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.113.148:33743
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786315935617919-15814-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786315935617919-15814-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.113.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786315935617919-15814-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.113.131
--webserver_port=0
--tserver_master_addrs=127.15.113.190:33897
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.113.131
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786315935617919-15814-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:57:22.909703 11324 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:57:22.911530 11324 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:57:22.924652 11331 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:57:22.924611 11334 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:57:22.925009 11336 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:57:22.925653 11324 server_base.cc:1034] running on GCE node
I20260809 22:57:22.926247 11324 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:57:22.927182 11324 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:57:22.928416 11324 hybrid_clock.cc:648] HybridClock initialized: now 1786316242928369 us; error 26 us; skew 500 ppm
I20260809 22:57:22.928783 11324 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 22:57:22.931039 11324 webserver.cc:460] Webserver started at http://127.15.113.131:34567/ using document root <none> and password file <none>
I20260809 22:57:22.931708 11324 fs_manager.cc:359] Metadata directory not provided
I20260809 22:57:22.931821 11324 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:57:22.932157 11324 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:57:22.935122 11324 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786315935617919-15814-0/minicluster-data/ts-2/data/instance:
uuid: "c9c6e21c652e4ea5b9908750e8599a32"
format_stamp: "Formatted at 2026-08-09 22:57:22 on dist-test-slave-ncp9"
server_key: "25d02d6e15654a47dbfbb20593c1874c"
server_key_iv: "6bde76f061ab8af022655d89eaa4e8e9"
server_key_version: "encryptionkey@0"
I20260809 22:57:22.935926 11324 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786315935617919-15814-0/minicluster-data/ts-2/wal/instance:
uuid: "c9c6e21c652e4ea5b9908750e8599a32"
format_stamp: "Formatted at 2026-08-09 22:57:22 on dist-test-slave-ncp9"
server_key: "25d02d6e15654a47dbfbb20593c1874c"
server_key_iv: "6bde76f061ab8af022655d89eaa4e8e9"
server_key_version: "encryptionkey@0"
I20260809 22:57:22.941080 11324 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.005s
I20260809 22:57:22.944450 11348 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:57:22.945699 11324 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 22:57:22.945856 11324 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786315935617919-15814-0/minicluster-data/ts-2/data,/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786315935617919-15814-0/minicluster-data/ts-2/wal
uuid: "c9c6e21c652e4ea5b9908750e8599a32"
format_stamp: "Formatted at 2026-08-09 22:57:22 on dist-test-slave-ncp9"
server_key: "25d02d6e15654a47dbfbb20593c1874c"
server_key_iv: "6bde76f061ab8af022655d89eaa4e8e9"
server_key_version: "encryptionkey@0"
I20260809 22:57:22.946058 11324 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786315935617919-15814-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786315935617919-15814-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786315935617919-15814-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:57:22.997684 11324 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:57:22.998500 11324 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:57:22.998790 11324 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:57:22.999732 11324 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:57:23.001210 11324 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:57:23.001305 11324 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:57:23.001405 11324 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:57:23.001473 11324 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:57:23.031191 11324 rpc_server.cc:307] RPC server started. Bound to: 127.15.113.131:36359
I20260809 22:57:23.031255 11515 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.113.131:36359 every 8 connection(s)
I20260809 22:57:23.032418 11324 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786315935617919-15814-0/minicluster-data/ts-2/data/info.pb
I20260809 22:57:23.037395 15814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskItYPJR/build/asan/bin/kudu as pid 11324
I20260809 22:57:23.037571 15814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786315935617919-15814-0/minicluster-data/ts-2/wal/instance
I20260809 22:57:23.038111 15814 external_mini_cluster.cc:1428] Setting key 0ffa07443f4f606df1d1982fb9ebad66
I20260809 22:57:23.045001 11133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:57:23.034892 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.113.131:38267 (local address 127.15.113.190:33897)
0809 22:57:23.035071 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0809 22:57:23.035076 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:57:23.036416 (+ 1340us) server_negotiation.cc:407] Connection header received
0809 22:57:23.038720 (+ 2304us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:57:23.038727 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:57:23.038820 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:57:23.038912 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:57:23.040284 (+ 1372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:57:23.041301 (+ 1017us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:57:23.042795 (+ 1494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:57:23.043170 (+ 375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:57:23.043257 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:57:23.044381 (+ 1124us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:57:23.044386 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:57:23.044390 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 22:57:23.044407 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:57:23.044439 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 22:57:23.044450 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:57:23.044514 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0809 22:57:23.044666 (+ 152us) server_negotiation.cc:299] Negotiation successful
0809 22:57:23.044738 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260809 22:57:23.045955 11523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:57:23.035386 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:33897 (local address 127.15.113.131:38267)
0809 22:57:23.036263 (+ 877us) negotiation.cc:107] Waiting for socket to connect
0809 22:57:23.036309 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0809 22:57:23.038402 (+ 2093us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:57:23.039190 (+ 788us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:57:23.039201 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:57:23.039320 (+ 119us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:57:23.040126 (+ 806us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:57:23.040147 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:57:23.041449 (+ 1302us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:57:23.041453 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:57:23.042685 (+ 1232us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:57:23.042698 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:57:23.043329 (+ 631us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:57:23.043334 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:57:23.043523 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:57:23.044100 (+ 577us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 22:57:23.044103 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:57:23.044140 (+ 37us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 22:57:23.044216 (+ 76us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:57:23.044520 (+ 304us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:57:23.044526 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:57:23.044530 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 22:57:23.044653 (+ 123us) client_negotiation.cc:240] Negotiation successful
0809 22:57:23.044973 (+ 320us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":441,"thread_start_us":255,"threads_started":1}
I20260809 22:57:23.047192 11517 heartbeater.cc:347] Connected to a master server at 127.15.113.190:33897
I20260809 22:57:23.047443 11517 heartbeater.cc:464] Registering TS with master...
I20260809 22:57:23.048098 11517 heartbeater.cc:511] Master 127.15.113.190:33897 requested a full tablet report, sending...
I20260809 22:57:23.049379 10835 ts_manager.cc:194] Registered new tserver with Master: c9c6e21c652e4ea5b9908750e8599a32 (127.15.113.131:36359)
I20260809 22:57:23.050166 10835 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.15.113.131:38267
I20260809 22:57:23.053076 15814 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 22:57:23.062664 11133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:57:23.057401 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35894 (local address 127.15.113.190:33897)
0809 22:57:23.057555 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0809 22:57:23.057559 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 22:57:23.058029 (+ 470us) server_negotiation.cc:407] Connection header received
0809 22:57:23.058150 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:57:23.058154 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:57:23.058222 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:57:23.058298 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:57:23.058767 (+ 469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:57:23.059431 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:57:23.060748 (+ 1317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:57:23.061172 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:57:23.061257 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:57:23.062158 (+ 901us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:57:23.062162 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:57:23.062165 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 22:57:23.062181 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:57:23.062225 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 22:57:23.062236 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:57:23.062300 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0809 22:57:23.062384 (+ 84us) server_negotiation.cc:299] Negotiation successful
0809 22:57:23.062449 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260809 22:57:23.065948 10835 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35894:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 22:57:23.072554 10835 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 22:57:23.093107 11538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:57:23.083633 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.129:37199 (local address 127.0.0.1:58792)
0809 22:57:23.084374 (+ 741us) negotiation.cc:107] Waiting for socket to connect
0809 22:57:23.084416 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0809 22:57:23.084612 (+ 196us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:57:23.085812 (+ 1200us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:57:23.085819 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:57:23.085863 (+ 44us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:57:23.086329 (+ 466us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:57:23.086345 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:57:23.088380 (+ 2035us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:57:23.088385 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:57:23.090540 (+ 2155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:57:23.090553 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:57:23.092135 (+ 1582us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:57:23.092142 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:57:23.092350 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:57:23.092400 (+ 50us) client_negotiation.cc:769] Sending connection context
0809 22:57:23.092516 (+ 116us) client_negotiation.cc:240] Negotiation successful
0809 22:57:23.092713 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":460,"spinlock_wait_cycles":6016,"thread_start_us":300,"threads_started":1}
I20260809 22:57:23.093086 11540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:57:23.083815 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58792 (local address 127.15.113.129:37199)
0809 22:57:23.085183 (+ 1368us) server_negotiation.cc:206] Beginning negotiation
0809 22:57:23.085190 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:57:23.085214 (+ 24us) server_negotiation.cc:407] Connection header received
0809 22:57:23.085291 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:57:23.085296 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:57:23.085559 (+ 263us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:57:23.086548 (+ 989us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:57:23.086805 (+ 257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:57:23.088242 (+ 1437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:57:23.090751 (+ 2509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:57:23.091912 (+ 1161us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:57:23.092074 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:57:23.092360 (+ 286us) server_negotiation.cc:1035] Waiting for connection context
0809 22:57:23.092570 (+ 210us) server_negotiation.cc:299] Negotiation successful
0809 22:57:23.092738 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1217,"thread_start_us":1094,"threads_started":1}
I20260809 22:57:23.094774 11541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:57:23.084791 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.130:37447 (local address 127.0.0.1:50048)
0809 22:57:23.085736 (+ 945us) negotiation.cc:107] Waiting for socket to connect
0809 22:57:23.085752 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 22:57:23.085846 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:57:23.087494 (+ 1648us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:57:23.087498 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:57:23.087524 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:57:23.088105 (+ 581us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:57:23.088115 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:57:23.089958 (+ 1843us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:57:23.089963 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:57:23.092143 (+ 2180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:57:23.092163 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:57:23.093980 (+ 1817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:57:23.093986 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:57:23.094163 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:57:23.094195 (+ 32us) client_negotiation.cc:769] Sending connection context
0809 22:57:23.094291 (+ 96us) client_negotiation.cc:240] Negotiation successful
0809 22:57:23.094421 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":810,"mutex_wait_us":34,"thread_start_us":1271,"threads_started":1}
I20260809 22:57:23.094851 11542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:57:23.085275 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50048 (local address 127.15.113.130:37447)
0809 22:57:23.086576 (+ 1301us) server_negotiation.cc:206] Beginning negotiation
0809 22:57:23.086585 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 22:57:23.086616 (+ 31us) server_negotiation.cc:407] Connection header received
0809 22:57:23.087055 (+ 439us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:57:23.087063 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:57:23.087287 (+ 224us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:57:23.087670 (+ 383us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:57:23.088264 (+ 594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:57:23.089814 (+ 1550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:57:23.092387 (+ 2573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:57:23.093798 (+ 1411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:57:23.093957 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:57:23.094174 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0809 22:57:23.094335 (+ 161us) server_negotiation.cc:299] Negotiation successful
0809 22:57:23.094498 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1153,"thread_start_us":1053,"threads_started":1}
I20260809 22:57:23.095110 11543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:57:23.085186 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.131:36359 (local address 127.0.0.1:39648)
0809 22:57:23.086316 (+ 1130us) negotiation.cc:107] Waiting for socket to connect
0809 22:57:23.086333 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 22:57:23.086433 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:57:23.087531 (+ 1098us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:57:23.087536 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:57:23.087562 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:57:23.087911 (+ 349us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:57:23.087923 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:57:23.089558 (+ 1635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:57:23.089564 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:57:23.092232 (+ 2668us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:57:23.092243 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:57:23.094313 (+ 2070us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:57:23.094319 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:57:23.094515 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:57:23.094546 (+ 31us) client_negotiation.cc:769] Sending connection context
0809 22:57:23.094636 (+ 90us) client_negotiation.cc:240] Negotiation successful
0809 22:57:23.094757 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1010,"mutex_wait_us":295,"thread_start_us":265,"threads_started":1}
I20260809 22:57:23.095865 11545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:57:23.085915 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39648 (local address 127.15.113.131:36359)
0809 22:57:23.086943 (+ 1028us) server_negotiation.cc:206] Beginning negotiation
0809 22:57:23.086951 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 22:57:23.086984 (+ 33us) server_negotiation.cc:407] Connection header received
0809 22:57:23.087087 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:57:23.087094 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:57:23.087327 (+ 233us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:57:23.088132 (+ 805us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:57:23.088406 (+ 274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:57:23.089426 (+ 1020us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:57:23.092422 (+ 2996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:57:23.093798 (+ 1376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:57:23.093986 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:57:23.094169 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0809 22:57:23.095473 (+ 1304us) server_negotiation.cc:299] Negotiation successful
0809 22:57:23.095615 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":867,"thread_start_us":721,"threads_started":1}
I20260809 22:57:23.097543 11014 tablet_service.cc:1467] Processing CreateTablet for tablet 91ba59014e8246688c29567f3280a403 (DEFAULT_TABLE table=test-table [id=89e091843fb24ee88e4d740110ffd6b5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:57:23.099038 11014 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 91ba59014e8246688c29567f3280a403. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:57:23.098603 11221 tablet_service.cc:1467] Processing CreateTablet for tablet 91ba59014e8246688c29567f3280a403 (DEFAULT_TABLE table=test-table [id=89e091843fb24ee88e4d740110ffd6b5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:57:23.098803 11418 tablet_service.cc:1467] Processing CreateTablet for tablet 91ba59014e8246688c29567f3280a403 (DEFAULT_TABLE table=test-table [id=89e091843fb24ee88e4d740110ffd6b5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:57:23.099936 11221 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 91ba59014e8246688c29567f3280a403. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:57:23.099946 11418 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 91ba59014e8246688c29567f3280a403. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:57:23.110845 11546 tablet_bootstrap.cc:492] T 91ba59014e8246688c29567f3280a403 P c9c6e21c652e4ea5b9908750e8599a32: Bootstrap starting.
I20260809 22:57:23.113651 11547 tablet_bootstrap.cc:492] T 91ba59014e8246688c29567f3280a403 P 135dbe5f19b04fa3a83a683b52dee272: Bootstrap starting.
I20260809 22:57:23.114215 11546 tablet_bootstrap.cc:654] T 91ba59014e8246688c29567f3280a403 P c9c6e21c652e4ea5b9908750e8599a32: Neither blocks nor log segments found. Creating new log.
I20260809 22:57:23.115794 11546 log.cc:824] T 91ba59014e8246688c29567f3280a403 P c9c6e21c652e4ea5b9908750e8599a32: Log is configured to *not* fsync() on all Append() calls
I20260809 22:57:23.117408 11547 tablet_bootstrap.cc:654] T 91ba59014e8246688c29567f3280a403 P 135dbe5f19b04fa3a83a683b52dee272: Neither blocks nor log segments found. Creating new log.
I20260809 22:57:23.118656 11547 log.cc:824] T 91ba59014e8246688c29567f3280a403 P 135dbe5f19b04fa3a83a683b52dee272: Log is configured to *not* fsync() on all Append() calls
I20260809 22:57:23.118906 11549 tablet_bootstrap.cc:492] T 91ba59014e8246688c29567f3280a403 P c5c76dc3b3ac445885bf42f67890c49b: Bootstrap starting.
I20260809 22:57:23.120217 11546 tablet_bootstrap.cc:492] T 91ba59014e8246688c29567f3280a403 P c9c6e21c652e4ea5b9908750e8599a32: No bootstrap required, opened a new log
I20260809 22:57:23.120571 11546 ts_tablet_manager.cc:1397] T 91ba59014e8246688c29567f3280a403 P c9c6e21c652e4ea5b9908750e8599a32: Time spent bootstrapping tablet: real 0.010s user 0.008s sys 0.000s
I20260809 22:57:23.122334 11549 tablet_bootstrap.cc:654] T 91ba59014e8246688c29567f3280a403 P c5c76dc3b3ac445885bf42f67890c49b: Neither blocks nor log segments found. Creating new log.
I20260809 22:57:23.124545 11549 log.cc:824] T 91ba59014e8246688c29567f3280a403 P c5c76dc3b3ac445885bf42f67890c49b: Log is configured to *not* fsync() on all Append() calls
I20260809 22:57:23.127560 11549 tablet_bootstrap.cc:492] T 91ba59014e8246688c29567f3280a403 P c5c76dc3b3ac445885bf42f67890c49b: No bootstrap required, opened a new log
I20260809 22:57:23.127562 11547 tablet_bootstrap.cc:492] T 91ba59014e8246688c29567f3280a403 P 135dbe5f19b04fa3a83a683b52dee272: No bootstrap required, opened a new log
I20260809 22:57:23.127825 11547 ts_tablet_manager.cc:1397] T 91ba59014e8246688c29567f3280a403 P 135dbe5f19b04fa3a83a683b52dee272: Time spent bootstrapping tablet: real 0.014s user 0.006s sys 0.006s
I20260809 22:57:23.127825 11549 ts_tablet_manager.cc:1397] T 91ba59014e8246688c29567f3280a403 P c5c76dc3b3ac445885bf42f67890c49b: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260809 22:57:23.129824 11546 raft_consensus.cc:348] T 91ba59014e8246688c29567f3280a403 P c9c6e21c652e4ea5b9908750e8599a32 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "135dbe5f19b04fa3a83a683b52dee272" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 37199 } } peers { permanent_uuid: "c5c76dc3b3ac445885bf42f67890c49b" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 37447 } } peers { permanent_uuid: "c9c6e21c652e4ea5b9908750e8599a32" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 36359 } }
I20260809 22:57:23.130151 11546 raft_consensus.cc:374] T 91ba59014e8246688c29567f3280a403 P c9c6e21c652e4ea5b9908750e8599a32 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:57:23.130245 11546 raft_consensus.cc:729] T 91ba59014e8246688c29567f3280a403 P c9c6e21c652e4ea5b9908750e8599a32 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c9c6e21c652e4ea5b9908750e8599a32, State: Initialized, Role: FOLLOWER
I20260809 22:57:23.130797 11546 consensus_queue.cc:260] T 91ba59014e8246688c29567f3280a403 P c9c6e21c652e4ea5b9908750e8599a32 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "135dbe5f19b04fa3a83a683b52dee272" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 37199 } } peers { permanent_uuid: "c5c76dc3b3ac445885bf42f67890c49b" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 37447 } } peers { permanent_uuid: "c9c6e21c652e4ea5b9908750e8599a32" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 36359 } }
I20260809 22:57:23.131768 11517 heartbeater.cc:503] Master 127.15.113.190:33897 was elected leader, sending a full tablet report...
I20260809 22:57:23.132411 11546 ts_tablet_manager.cc:1428] T 91ba59014e8246688c29567f3280a403 P c9c6e21c652e4ea5b9908750e8599a32: Time spent starting tablet: real 0.012s user 0.004s sys 0.006s
I20260809 22:57:23.136654 11130 heartbeater.cc:503] Master 127.15.113.190:33897 was elected leader, sending a full tablet report...
I20260809 22:57:23.137948 11547 raft_consensus.cc:348] T 91ba59014e8246688c29567f3280a403 P 135dbe5f19b04fa3a83a683b52dee272 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "135dbe5f19b04fa3a83a683b52dee272" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 37199 } } peers { permanent_uuid: "c5c76dc3b3ac445885bf42f67890c49b" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 37447 } } peers { permanent_uuid: "c9c6e21c652e4ea5b9908750e8599a32" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 36359 } }
I20260809 22:57:23.137946 11549 raft_consensus.cc:348] T 91ba59014e8246688c29567f3280a403 P c5c76dc3b3ac445885bf42f67890c49b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "135dbe5f19b04fa3a83a683b52dee272" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 37199 } } peers { permanent_uuid: "c5c76dc3b3ac445885bf42f67890c49b" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 37447 } } peers { permanent_uuid: "c9c6e21c652e4ea5b9908750e8599a32" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 36359 } }
I20260809 22:57:23.138367 11547 raft_consensus.cc:374] T 91ba59014e8246688c29567f3280a403 P 135dbe5f19b04fa3a83a683b52dee272 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:57:23.138460 11549 raft_consensus.cc:374] T 91ba59014e8246688c29567f3280a403 P c5c76dc3b3ac445885bf42f67890c49b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:57:23.138473 11547 raft_consensus.cc:729] T 91ba59014e8246688c29567f3280a403 P 135dbe5f19b04fa3a83a683b52dee272 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 135dbe5f19b04fa3a83a683b52dee272, State: Initialized, Role: FOLLOWER
I20260809 22:57:23.138556 11549 raft_consensus.cc:729] T 91ba59014e8246688c29567f3280a403 P c5c76dc3b3ac445885bf42f67890c49b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c5c76dc3b3ac445885bf42f67890c49b, State: Initialized, Role: FOLLOWER
I20260809 22:57:23.139075 11547 consensus_queue.cc:260] T 91ba59014e8246688c29567f3280a403 P 135dbe5f19b04fa3a83a683b52dee272 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "135dbe5f19b04fa3a83a683b52dee272" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 37199 } } peers { permanent_uuid: "c5c76dc3b3ac445885bf42f67890c49b" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 37447 } } peers { permanent_uuid: "c9c6e21c652e4ea5b9908750e8599a32" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 36359 } }
I20260809 22:57:23.139138 11549 consensus_queue.cc:260] T 91ba59014e8246688c29567f3280a403 P c5c76dc3b3ac445885bf42f67890c49b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "135dbe5f19b04fa3a83a683b52dee272" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 37199 } } peers { permanent_uuid: "c5c76dc3b3ac445885bf42f67890c49b" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 37447 } } peers { permanent_uuid: "c9c6e21c652e4ea5b9908750e8599a32" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 36359 } }
I20260809 22:57:23.140504 11321 heartbeater.cc:503] Master 127.15.113.190:33897 was elected leader, sending a full tablet report...
I20260809 22:57:23.141100 11549 ts_tablet_manager.cc:1428] T 91ba59014e8246688c29567f3280a403 P c5c76dc3b3ac445885bf42f67890c49b: Time spent starting tablet: real 0.013s user 0.014s sys 0.001s
I20260809 22:57:23.141876 11547 ts_tablet_manager.cc:1428] T 91ba59014e8246688c29567f3280a403 P 135dbe5f19b04fa3a83a683b52dee272: Time spent starting tablet: real 0.014s user 0.007s sys 0.007s
I20260809 22:57:23.203367 11558 raft_consensus.cc:482] T 91ba59014e8246688c29567f3280a403 P c5c76dc3b3ac445885bf42f67890c49b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 22:57:23.203681 11558 raft_consensus.cc:504] T 91ba59014e8246688c29567f3280a403 P c5c76dc3b3ac445885bf42f67890c49b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "135dbe5f19b04fa3a83a683b52dee272" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 37199 } } peers { permanent_uuid: "c5c76dc3b3ac445885bf42f67890c49b" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 37447 } } peers { permanent_uuid: "c9c6e21c652e4ea5b9908750e8599a32" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 36359 } }
I20260809 22:57:23.205483 11558 leader_election.cc:288] T 91ba59014e8246688c29567f3280a403 P c5c76dc3b3ac445885bf42f67890c49b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 135dbe5f19b04fa3a83a683b52dee272 (127.15.113.129:37199), c9c6e21c652e4ea5b9908750e8599a32 (127.15.113.131:36359)
I20260809 22:57:23.214075 11545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:57:23.205960 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.113.130:57071 (local address 127.15.113.131:36359)
0809 22:57:23.206133 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0809 22:57:23.206138 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:57:23.206694 (+ 556us) server_negotiation.cc:407] Connection header received
0809 22:57:23.206871 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:57:23.206876 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:57:23.206956 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:57:23.207058 (+ 102us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:57:23.207831 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:57:23.208856 (+ 1025us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:57:23.211799 (+ 2943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:57:23.213023 (+ 1224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:57:23.213170 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:57:23.213241 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0809 22:57:23.213698 (+ 457us) server_negotiation.cc:299] Negotiation successful
0809 22:57:23.213830 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260809 22:57:23.214075 11540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:57:23.205824 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.113.130:37831 (local address 127.15.113.129:37199)
0809 22:57:23.206010 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0809 22:57:23.206017 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:57:23.206688 (+ 671us) server_negotiation.cc:407] Connection header received
0809 22:57:23.206897 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:57:23.206902 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:57:23.206985 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:57:23.207068 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:57:23.207755 (+ 687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:57:23.208837 (+ 1082us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:57:23.211799 (+ 2962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:57:23.213011 (+ 1212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:57:23.213171 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:57:23.213241 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0809 22:57:23.213698 (+ 457us) server_negotiation.cc:299] Negotiation successful
0809 22:57:23.213830 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260809 22:57:23.214320 11562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:57:23.205902 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.131:36359 (local address 127.15.113.130:57071)
0809 22:57:23.206592 (+ 690us) negotiation.cc:107] Waiting for socket to connect
0809 22:57:23.206607 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 22:57:23.206737 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:57:23.207072 (+ 335us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:57:23.207077 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:57:23.207119 (+ 42us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:57:23.207681 (+ 562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:57:23.207692 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:57:23.209007 (+ 1315us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:57:23.209013 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:57:23.211585 (+ 2572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:57:23.211594 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:57:23.213213 (+ 1619us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:57:23.213219 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:57:23.213479 (+ 260us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:57:23.213571 (+ 92us) client_negotiation.cc:769] Sending connection context
0809 22:57:23.213662 (+ 91us) client_negotiation.cc:240] Negotiation successful
0809 22:57:23.213852 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":591,"mutex_wait_us":307,"spinlock_wait_cycles":15488,"thread_start_us":390,"threads_started":1}
I20260809 22:57:23.214425 11559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:57:23.205794 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.129:37199 (local address 127.15.113.130:37831)
0809 22:57:23.206555 (+ 761us) negotiation.cc:107] Waiting for socket to connect
0809 22:57:23.206584 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0809 22:57:23.206778 (+ 194us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:57:23.207078 (+ 300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:57:23.207082 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:57:23.207121 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:57:23.207605 (+ 484us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:57:23.207623 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:57:23.209008 (+ 1385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:57:23.209013 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:57:23.211577 (+ 2564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:57:23.211594 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:57:23.213221 (+ 1627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:57:23.213227 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:57:23.213479 (+ 252us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:57:23.213568 (+ 89us) client_negotiation.cc:769] Sending connection context
0809 22:57:23.213662 (+ 94us) client_negotiation.cc:240] Negotiation successful
0809 22:57:23.213852 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":601,"mutex_wait_us":217,"spinlock_wait_cycles":39296,"thread_start_us":393,"threads_started":1}
I20260809 22:57:23.215263 11449 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "91ba59014e8246688c29567f3280a403" candidate_uuid: "c5c76dc3b3ac445885bf42f67890c49b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c9c6e21c652e4ea5b9908750e8599a32" is_pre_election: true
I20260809 22:57:23.215263 11047 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "91ba59014e8246688c29567f3280a403" candidate_uuid: "c5c76dc3b3ac445885bf42f67890c49b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "135dbe5f19b04fa3a83a683b52dee272" is_pre_election: true
I20260809 22:57:23.215758 11449 raft_consensus.cc:2454] T 91ba59014e8246688c29567f3280a403 P c9c6e21c652e4ea5b9908750e8599a32 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c5c76dc3b3ac445885bf42f67890c49b in term 0.
I20260809 22:57:23.215755 11047 raft_consensus.cc:2454] T 91ba59014e8246688c29567f3280a403 P 135dbe5f19b04fa3a83a683b52dee272 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c5c76dc3b3ac445885bf42f67890c49b in term 0.
I20260809 22:57:23.216356 11165 leader_election.cc:302] T 91ba59014e8246688c29567f3280a403 P c5c76dc3b3ac445885bf42f67890c49b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c5c76dc3b3ac445885bf42f67890c49b, c9c6e21c652e4ea5b9908750e8599a32; no voters:
I20260809 22:57:23.216730 11558 raft_consensus.cc:2781] T 91ba59014e8246688c29567f3280a403 P c5c76dc3b3ac445885bf42f67890c49b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 22:57:23.216878 11558 raft_consensus.cc:482] T 91ba59014e8246688c29567f3280a403 P c5c76dc3b3ac445885bf42f67890c49b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 22:57:23.217005 11558 raft_consensus.cc:3037] T 91ba59014e8246688c29567f3280a403 P c5c76dc3b3ac445885bf42f67890c49b [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:57:23.219137 11558 raft_consensus.cc:504] T 91ba59014e8246688c29567f3280a403 P c5c76dc3b3ac445885bf42f67890c49b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "135dbe5f19b04fa3a83a683b52dee272" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 37199 } } peers { permanent_uuid: "c5c76dc3b3ac445885bf42f67890c49b" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 37447 } } peers { permanent_uuid: "c9c6e21c652e4ea5b9908750e8599a32" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 36359 } }
I20260809 22:57:23.219648 11558 leader_election.cc:288] T 91ba59014e8246688c29567f3280a403 P c5c76dc3b3ac445885bf42f67890c49b [CANDIDATE]: Term 1 election: Requested vote from peers 135dbe5f19b04fa3a83a683b52dee272 (127.15.113.129:37199), c9c6e21c652e4ea5b9908750e8599a32 (127.15.113.131:36359)
I20260809 22:57:23.220083 11047 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "91ba59014e8246688c29567f3280a403" candidate_uuid: "c5c76dc3b3ac445885bf42f67890c49b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "135dbe5f19b04fa3a83a683b52dee272"
I20260809 22:57:23.220166 11449 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "91ba59014e8246688c29567f3280a403" candidate_uuid: "c5c76dc3b3ac445885bf42f67890c49b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c9c6e21c652e4ea5b9908750e8599a32"
I20260809 22:57:23.220311 11047 raft_consensus.cc:3037] T 91ba59014e8246688c29567f3280a403 P 135dbe5f19b04fa3a83a683b52dee272 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:57:23.220353 11449 raft_consensus.cc:3037] T 91ba59014e8246688c29567f3280a403 P c9c6e21c652e4ea5b9908750e8599a32 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:57:23.223503 11449 raft_consensus.cc:2454] T 91ba59014e8246688c29567f3280a403 P c9c6e21c652e4ea5b9908750e8599a32 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c5c76dc3b3ac445885bf42f67890c49b in term 1.
I20260809 22:57:23.223503 11047 raft_consensus.cc:2454] T 91ba59014e8246688c29567f3280a403 P 135dbe5f19b04fa3a83a683b52dee272 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c5c76dc3b3ac445885bf42f67890c49b in term 1.
I20260809 22:57:23.223875 11164 leader_election.cc:302] T 91ba59014e8246688c29567f3280a403 P c5c76dc3b3ac445885bf42f67890c49b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 135dbe5f19b04fa3a83a683b52dee272, c5c76dc3b3ac445885bf42f67890c49b; no voters:
I20260809 22:57:23.224211 11558 raft_consensus.cc:2781] T 91ba59014e8246688c29567f3280a403 P c5c76dc3b3ac445885bf42f67890c49b [term 1 FOLLOWER]: Leader election won for term 1
I20260809 22:57:23.224618 11558 raft_consensus.cc:686] T 91ba59014e8246688c29567f3280a403 P c5c76dc3b3ac445885bf42f67890c49b [term 1 LEADER]: Becoming Leader. State: Replica: c5c76dc3b3ac445885bf42f67890c49b, State: Running, Role: LEADER
I20260809 22:57:23.225112 11558 consensus_queue.cc:237] T 91ba59014e8246688c29567f3280a403 P c5c76dc3b3ac445885bf42f67890c49b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "135dbe5f19b04fa3a83a683b52dee272" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 37199 } } peers { permanent_uuid: "c5c76dc3b3ac445885bf42f67890c49b" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 37447 } } peers { permanent_uuid: "c9c6e21c652e4ea5b9908750e8599a32" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 36359 } }
I20260809 22:57:23.229338 10835 catalog_manager.cc:5495] T 91ba59014e8246688c29567f3280a403 P c5c76dc3b3ac445885bf42f67890c49b reported cstate change: term changed from 0 to 1, leader changed from <none> to c5c76dc3b3ac445885bf42f67890c49b (127.15.113.130). New cstate: current_term: 1 leader_uuid: "c5c76dc3b3ac445885bf42f67890c49b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "135dbe5f19b04fa3a83a683b52dee272" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 37199 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c5c76dc3b3ac445885bf42f67890c49b" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 37447 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c9c6e21c652e4ea5b9908750e8599a32" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 36359 } health_report { overall_health: UNKNOWN } } }
I20260809 22:57:23.245062 11542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:57:23.240395 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50058 (local address 127.15.113.130:37447)
0809 22:57:23.240551 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0809 22:57:23.240556 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:57:23.240574 (+ 18us) server_negotiation.cc:407] Connection header received
0809 22:57:23.240762 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:57:23.240765 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:57:23.240852 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:57:23.240995 (+ 143us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 22:57:23.241280 (+ 285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:57:23.241944 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:57:23.243293 (+ 1349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:57:23.243692 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:57:23.243780 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:57:23.244168 (+ 388us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 22:57:23.244626 (+ 458us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 22:57:23.244731 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0809 22:57:23.244772 (+ 41us) server_negotiation.cc:299] Negotiation successful
0809 22:57:23.244831 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260809 22:57:23.252525 11449 raft_consensus.cc:1261] T 91ba59014e8246688c29567f3280a403 P c9c6e21c652e4ea5b9908750e8599a32 [term 1 FOLLOWER]: Refusing update from remote peer c5c76dc3b3ac445885bf42f67890c49b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 22:57:23.252601 11047 raft_consensus.cc:1261] T 91ba59014e8246688c29567f3280a403 P 135dbe5f19b04fa3a83a683b52dee272 [term 1 FOLLOWER]: Refusing update from remote peer c5c76dc3b3ac445885bf42f67890c49b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 22:57:23.253437 11563 consensus_queue.cc:1035] T 91ba59014e8246688c29567f3280a403 P c5c76dc3b3ac445885bf42f67890c49b [LEADER]: Connected to new peer: Peer: permanent_uuid: "c9c6e21c652e4ea5b9908750e8599a32" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 36359 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 22:57:23.253739 11558 consensus_queue.cc:1035] T 91ba59014e8246688c29567f3280a403 P c5c76dc3b3ac445885bf42f67890c49b [LEADER]: Connected to new peer: Peer: permanent_uuid: "135dbe5f19b04fa3a83a683b52dee272" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 37199 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 22:57:23.264030 11570 mvcc.cc:205] Tried to move back new op lower bound from 7316751332354109440 to 7316751332253564928. Current Snapshot: MvccSnapshot[applied={T|T < 7316751332354109440}]
I20260809 22:57:23.278927 10835 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:35894:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 22:57:23.279129 10835 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:35894:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 22:57:23.282469 10835 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3d79d7fea78343d1b189cbadad6cf10e: Sending DeleteTablet for 3 replicas of tablet 91ba59014e8246688c29567f3280a403
I20260809 22:57:23.283519 11014 tablet_service.cc:1514] Processing DeleteTablet for tablet 91ba59014e8246688c29567f3280a403 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:57:23 UTC) from {username='slave'} at 127.0.0.1:58792
I20260809 22:57:23.283589 11221 tablet_service.cc:1514] Processing DeleteTablet for tablet 91ba59014e8246688c29567f3280a403 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:57:23 UTC) from {username='slave'} at 127.0.0.1:50048
I20260809 22:57:23.283876 11418 tablet_service.cc:1514] Processing DeleteTablet for tablet 91ba59014e8246688c29567f3280a403 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:57:23 UTC) from {username='slave'} at 127.0.0.1:39648
I20260809 22:57:23.285684 15814 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskItYPJR/build/asan/bin/kudu pid 10920
I20260809 22:57:23.295867 11588 tablet_replica.cc:330] T 91ba59014e8246688c29567f3280a403 P c9c6e21c652e4ea5b9908750e8599a32: stopping tablet replica
I20260809 22:57:23.296283 11588 raft_consensus.cc:2229] T 91ba59014e8246688c29567f3280a403 P c9c6e21c652e4ea5b9908750e8599a32 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 22:57:23.296839 11588 raft_consensus.cc:2258] T 91ba59014e8246688c29567f3280a403 P c9c6e21c652e4ea5b9908750e8599a32 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 22:57:23.298256 11587 tablet_replica.cc:330] T 91ba59014e8246688c29567f3280a403 P 135dbe5f19b04fa3a83a683b52dee272: stopping tablet replica
I20260809 22:57:23.298718 11587 raft_consensus.cc:2229] T 91ba59014e8246688c29567f3280a403 P 135dbe5f19b04fa3a83a683b52dee272 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 22:57:23.299294 11587 raft_consensus.cc:2258] T 91ba59014e8246688c29567f3280a403 P 135dbe5f19b04fa3a83a683b52dee272 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 22:57:23.301599 11587 ts_tablet_manager.cc:1905] T 91ba59014e8246688c29567f3280a403 P 135dbe5f19b04fa3a83a683b52dee272: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 22:57:23.303592 11586 tablet_replica.cc:330] T 91ba59014e8246688c29567f3280a403 P c5c76dc3b3ac445885bf42f67890c49b: stopping tablet replica
I20260809 22:57:23.304070 11586 raft_consensus.cc:2229] T 91ba59014e8246688c29567f3280a403 P c5c76dc3b3ac445885bf42f67890c49b [term 1 LEADER]: Raft consensus shutting down.
I20260809 22:57:23.304922 11588 ts_tablet_manager.cc:1905] T 91ba59014e8246688c29567f3280a403 P c9c6e21c652e4ea5b9908750e8599a32: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 22:57:23.307120 11540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:57:23.287079 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58800 (local address 127.15.113.129:37199)
0809 22:57:23.287272 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0809 22:57:23.287279 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:57:23.297632 (+ 10353us) server_negotiation.cc:407] Connection header received
0809 22:57:23.297800 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:57:23.297805 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:57:23.297935 (+ 130us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:57:23.298047 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:57:23.298748 (+ 701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:57:23.299929 (+ 1181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:57:23.303080 (+ 3151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:57:23.303766 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:57:23.303872 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:57:23.305460 (+ 1588us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:57:23.305471 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:57:23.305486 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 22:57:23.306211 (+ 725us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:57:23.306268 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 22:57:23.306297 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:57:23.306382 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0809 22:57:23.306639 (+ 257us) server_negotiation.cc:299] Negotiation successful
0809 22:57:23.306752 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260809 22:57:23.308480 11121 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:57:23.309265 11586 raft_consensus.cc:2258] T 91ba59014e8246688c29567f3280a403 P c5c76dc3b3ac445885bf42f67890c49b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 22:57:23.311580 11586 ts_tablet_manager.cc:1905] T 91ba59014e8246688c29567f3280a403 P c5c76dc3b3ac445885bf42f67890c49b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 22:57:23.312176 11588 ts_tablet_manager.cc:1918] T 91ba59014e8246688c29567f3280a403 P c9c6e21c652e4ea5b9908750e8599a32: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 22:57:23.312454 11588 log.cc:1196] T 91ba59014e8246688c29567f3280a403 P c9c6e21c652e4ea5b9908750e8599a32: Deleting WAL directory at /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786315935617919-15814-0/minicluster-data/ts-2/wal/wals/91ba59014e8246688c29567f3280a403
I20260809 22:57:23.313109 11588 ts_tablet_manager.cc:1939] T 91ba59014e8246688c29567f3280a403 P c9c6e21c652e4ea5b9908750e8599a32: Deleting consensus metadata
I20260809 22:57:23.315366 10816 catalog_manager.cc:4841] TS c9c6e21c652e4ea5b9908750e8599a32 (127.15.113.131:36359): tablet 91ba59014e8246688c29567f3280a403 (table test-table [id=89e091843fb24ee88e4d740110ffd6b5]) successfully deleted
I20260809 22:57:23.318416 11586 ts_tablet_manager.cc:1918] T 91ba59014e8246688c29567f3280a403 P c5c76dc3b3ac445885bf42f67890c49b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 22:57:23.318567 11586 log.cc:1196] T 91ba59014e8246688c29567f3280a403 P c5c76dc3b3ac445885bf42f67890c49b: Deleting WAL directory at /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786315935617919-15814-0/minicluster-data/ts-1/wal/wals/91ba59014e8246688c29567f3280a403
I20260809 22:57:23.319111 11586 ts_tablet_manager.cc:1939] T 91ba59014e8246688c29567f3280a403 P c5c76dc3b3ac445885bf42f67890c49b: Deleting consensus metadata
I20260809 22:57:23.320667 10815 catalog_manager.cc:4841] TS c5c76dc3b3ac445885bf42f67890c49b (127.15.113.130:37447): tablet 91ba59014e8246688c29567f3280a403 (table test-table [id=89e091843fb24ee88e4d740110ffd6b5]) successfully deleted
I20260809 22:57:23.402448 11587 ts_tablet_manager.cc:1918] T 91ba59014e8246688c29567f3280a403 P 135dbe5f19b04fa3a83a683b52dee272: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 22:57:23.402710 11587 log.cc:1196] T 91ba59014e8246688c29567f3280a403 P 135dbe5f19b04fa3a83a683b52dee272: Deleting WAL directory at /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786315935617919-15814-0/minicluster-data/ts-0/wal/wals/91ba59014e8246688c29567f3280a403
I20260809 22:57:23.403208 15814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskItYPJR/build/asan/bin/kudu with pid 10920
W20260809 22:57:23.431002 10815 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.15.113.129:37199 (error 108)
I20260809 22:57:23.431067 15814 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskItYPJR/build/asan/bin/kudu pid 11136
I20260809 22:57:23.433018 11543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:57:23.432519 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.129:37199 (local address 127.0.0.1:58804)
0809 22:57:23.432681 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0809 22:57:23.432817 (+ 136us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.113.129:37199: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260809 22:57:23.433388 10815 catalog_manager.cc:4584] TS 135dbe5f19b04fa3a83a683b52dee272 (127.15.113.129:37199): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 91ba59014e8246688c29567f3280a403: Network error: Client connection negotiation failed: client connection to 127.15.113.129:37199: connect: Connection refused (error 111)
I20260809 22:57:23.438681 11542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:57:23.432152 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50064 (local address 127.15.113.130:37447)
0809 22:57:23.432330 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0809 22:57:23.432336 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:57:23.432355 (+ 19us) server_negotiation.cc:407] Connection header received
0809 22:57:23.432427 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:57:23.432433 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:57:23.432534 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:57:23.432637 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:57:23.433276 (+ 639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:57:23.434217 (+ 941us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:57:23.435480 (+ 1263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:57:23.435926 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:57:23.436038 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:57:23.437213 (+ 1175us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:57:23.437222 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:57:23.437237 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 22:57:23.437914 (+ 677us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:57:23.437965 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 22:57:23.437989 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:57:23.438104 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0809 22:57:23.438249 (+ 145us) server_negotiation.cc:299] Negotiation successful
0809 22:57:23.438357 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260809 22:57:23.439460 11315 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:57:23.487221 11543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:57:23.486736 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.129:37199 (local address 127.0.0.1:58816)
0809 22:57:23.486943 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0809 22:57:23.487044 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.113.129:37199: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
I20260809 22:57:23.488199 11543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:57:23.487877 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.129:37199 (local address 127.0.0.1:58830)
0809 22:57:23.487993 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0809 22:57:23.488061 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.113.129:37199: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260809 22:57:23.548691 11543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:57:23.548201 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.129:37199 (local address 127.0.0.1:58846)
0809 22:57:23.548432 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0809 22:57:23.548521 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.113.129:37199: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81}
I20260809 22:57:23.550349 11543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:57:23.549928 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.129:37199 (local address 127.0.0.1:58850)
0809 22:57:23.550105 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0809 22:57:23.550209 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.113.129:37199: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260809 22:57:23.551159 15814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskItYPJR/build/asan/bin/kudu with pid 11136
I20260809 22:57:23.573493 15814 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskItYPJR/build/asan/bin/kudu pid 11324
I20260809 22:57:23.579983 11545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:57:23.574131 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39660 (local address 127.15.113.131:36359)
0809 22:57:23.574280 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0809 22:57:23.574284 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 22:57:23.574327 (+ 43us) server_negotiation.cc:407] Connection header received
0809 22:57:23.574486 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:57:23.574490 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:57:23.574572 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:57:23.574655 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:57:23.575001 (+ 346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:57:23.575716 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:57:23.577036 (+ 1320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:57:23.577441 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:57:23.577558 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:57:23.578607 (+ 1049us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:57:23.578618 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:57:23.578636 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 22:57:23.579328 (+ 692us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:57:23.579385 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 22:57:23.579409 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:57:23.579514 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0809 22:57:23.579577 (+ 63us) server_negotiation.cc:299] Negotiation successful
0809 22:57:23.579679 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260809 22:57:23.580646 11506 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:57:23.631609 11543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:57:23.631059 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.129:37199 (local address 127.0.0.1:58860)
0809 22:57:23.631308 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0809 22:57:23.631424 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.113.129:37199: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89}
I20260809 22:57:23.632726 11543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:57:23.632298 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.129:37199 (local address 127.0.0.1:58862)
0809 22:57:23.632498 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0809 22:57:23.632586 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.113.129:37199: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98}
I20260809 22:57:23.673658 15814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskItYPJR/build/asan/bin/kudu with pid 11324
I20260809 22:57:23.696179 15814 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskItYPJR/build/asan/bin/kudu pid 10792
I20260809 22:57:23.697243 10874 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:57:23.804167 15814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskItYPJR/build/asan/bin/kudu with pid 10792
2026-08-09T22:57:23Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (2759 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315935617919-15814-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 22:57:23 dist-test-slave-ncp9 krb5kdc[11600](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 22:57:23 dist-test-slave-ncp9 krb5kdc[11600](info): set up 2 sockets
Aug 09 22:57:23 dist-test-slave-ncp9 krb5kdc[11600](info): commencing operation
krb5kdc: starting...
W20260809 22:57:25.022025 15814 mini_kdc.cc:121] Time spent starting KDC: real 1.170s user 0.000s sys 0.041s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315935617919-15814-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315935617919-15814-0/krb5kdc/test-user.keytab.
Aug 09 22:57:25 dist-test-slave-ncp9 krb5kdc[11600](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316245, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T22:57:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T22:57:25Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.113.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.113.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.113.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.190.keytab.
Entry for principal kudu/127.15.113.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.190.keytab.
I20260809 22:57:25.291893 15814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskItYPJR/build/asan/bin/kudu
/tmp/dist-test-taskItYPJR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315935617919-15814-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315935617919-15814-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315935617919-15814-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315935617919-15814-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.113.190:36003
--webserver_interface=127.15.113.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.190.keytab
--principal=kudu/127.15.113.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.113.148:46077
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.113.190:36003
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315935617919-15814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315935617919-15814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315935617919-15814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:57:25.616282 11615 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:57:25.616565 11615 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:57:25.616686 11615 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:57:25.623770 11615 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 22:57:25.623869 11615 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:57:25.623945 11615 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:57:25.624020 11615 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 22:57:25.624082 11615 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 22:57:25.633562 11615 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.113.148:46077
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315935617919-15814-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315935617919-15814-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.113.190:36003
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.190.keytab
--principal=kudu/127.15.113.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.113.190:36003
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315935617919-15814-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.113.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315935617919-15814-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:57:25.634831 11615 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:57:25.636160 11615 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:57:25.647336 11621 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:57:25.647365 11625 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:57:25.647418 11623 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:57:25.648442 11615 server_base.cc:1034] running on GCE node
I20260809 22:57:25.649212 11615 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:57:25.650513 11615 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:57:25.651713 11615 hybrid_clock.cc:648] HybridClock initialized: now 1786316245651659 us; error 33 us; skew 500 ppm
I20260809 22:57:25.652033 11615 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:57:25 dist-test-slave-ncp9 krb5kdc[11600](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316245, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.113.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:57:25.660066 11615 init.cc:377] Logged in from keytab as kudu/127.15.113.190@KRBTEST.COM (short username kudu)
I20260809 22:57:25.661582 11615 webserver.cc:460] Webserver started at http://127.15.113.190:37991/ using document root <none> and password file <none>
I20260809 22:57:25.662276 11615 fs_manager.cc:359] Metadata directory not provided
I20260809 22:57:25.662391 11615 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:57:25.662695 11615 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:57:25.665630 11615 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315935617919-15814-0/minicluster-data/master-0/data/instance:
uuid: "a6315b057ded4e9cb09a53abc79dd676"
format_stamp: "Formatted at 2026-08-09 22:57:25 on dist-test-slave-ncp9"
server_key: "aea908aacf8e40cff9721677364b83a4"
server_key_iv: "c008e04516a25755b48b1d4d83c3797f"
server_key_version: "encryptionkey@0"
I20260809 22:57:25.666363 11615 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315935617919-15814-0/minicluster-data/master-0/wal/instance:
uuid: "a6315b057ded4e9cb09a53abc79dd676"
format_stamp: "Formatted at 2026-08-09 22:57:25 on dist-test-slave-ncp9"
server_key: "aea908aacf8e40cff9721677364b83a4"
server_key_iv: "c008e04516a25755b48b1d4d83c3797f"
server_key_version: "encryptionkey@0"
I20260809 22:57:25.671277 11615 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 22:57:25.674611 11639 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:57:25.676250 11615 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 22:57:25.676427 11615 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315935617919-15814-0/minicluster-data/master-0/data,/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315935617919-15814-0/minicluster-data/master-0/wal
uuid: "a6315b057ded4e9cb09a53abc79dd676"
format_stamp: "Formatted at 2026-08-09 22:57:25 on dist-test-slave-ncp9"
server_key: "aea908aacf8e40cff9721677364b83a4"
server_key_iv: "c008e04516a25755b48b1d4d83c3797f"
server_key_version: "encryptionkey@0"
I20260809 22:57:25.676648 11615 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315935617919-15814-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315935617919-15814-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315935617919-15814-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:57:25.687879 11615 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:57:25.691424 11615 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:57:25.691669 11615 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:57:25.708619 11615 rpc_server.cc:307] RPC server started. Bound to: 127.15.113.190:36003
I20260809 22:57:25.708639 11709 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.113.190:36003 every 8 connection(s)
I20260809 22:57:25.709965 11615 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315935617919-15814-0/minicluster-data/master-0/data/info.pb
I20260809 22:57:25.713624 11710 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:57:25.713826 15814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskItYPJR/build/asan/bin/kudu as pid 11615
I20260809 22:57:25.714025 15814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315935617919-15814-0/minicluster-data/master-0/wal/instance
I20260809 22:57:25.714582 15814 external_mini_cluster.cc:1428] Setting key 84832280e5a46ae5d3583c5d1c61a98e
I20260809 22:57:25.727181 11710 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a6315b057ded4e9cb09a53abc79dd676: Bootstrap starting.
Aug 09 22:57:25 dist-test-slave-ncp9 krb5kdc[11600](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316245, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.113.190@KRBTEST.COM
I20260809 22:57:25.730988 11710 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a6315b057ded4e9cb09a53abc79dd676: Neither blocks nor log segments found. Creating new log.
I20260809 22:57:25.732035 11710 log.cc:824] T 00000000000000000000000000000000 P a6315b057ded4e9cb09a53abc79dd676: Log is configured to *not* fsync() on all Append() calls
I20260809 22:57:25.735041 11710 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a6315b057ded4e9cb09a53abc79dd676: No bootstrap required, opened a new log
I20260809 22:57:25.738246 11713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:57:25.716481 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45480 (local address 127.15.113.190:36003)
0809 22:57:25.717296 (+ 815us) server_negotiation.cc:206] Beginning negotiation
0809 22:57:25.717307 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 22:57:25.717348 (+ 41us) server_negotiation.cc:407] Connection header received
0809 22:57:25.719656 (+ 2308us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:57:25.719679 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:57:25.720050 (+ 371us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:57:25.720378 (+ 328us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:57:25.722202 (+ 1824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:57:25.723528 (+ 1326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:57:25.725411 (+ 1883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:57:25.726053 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:57:25.726190 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:57:25.731078 (+ 4888us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:57:25.731088 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:57:25.731107 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:57:25.731154 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:57:25.734264 (+ 3110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:57:25.735046 (+ 782us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:57:25.735051 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:57:25.735057 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:57:25.735249 (+ 192us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:57:25.735672 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:57:25.735675 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:57:25.735677 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:57:25.736176 (+ 499us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:57:25.736417 (+ 241us) server_negotiation.cc:1035] Waiting for connection context
0809 22:57:25.736856 (+ 439us) server_negotiation.cc:299] Negotiation successful
0809 22:57:25.737275 (+ 419us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":585,"thread_start_us":379,"threads_started":1}
I20260809 22:57:25.742889 11710 raft_consensus.cc:348] T 00000000000000000000000000000000 P a6315b057ded4e9cb09a53abc79dd676 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a6315b057ded4e9cb09a53abc79dd676" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 36003 } }
I20260809 22:57:25.743261 11710 raft_consensus.cc:374] T 00000000000000000000000000000000 P a6315b057ded4e9cb09a53abc79dd676 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:57:25.743351 11710 raft_consensus.cc:729] T 00000000000000000000000000000000 P a6315b057ded4e9cb09a53abc79dd676 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a6315b057ded4e9cb09a53abc79dd676, State: Initialized, Role: FOLLOWER
I20260809 22:57:25.743952 11710 consensus_queue.cc:260] T 00000000000000000000000000000000 P a6315b057ded4e9cb09a53abc79dd676 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a6315b057ded4e9cb09a53abc79dd676" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 36003 } }
I20260809 22:57:25.744182 11710 raft_consensus.cc:388] T 00000000000000000000000000000000 P a6315b057ded4e9cb09a53abc79dd676 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 22:57:25.744307 11710 raft_consensus.cc:482] T 00000000000000000000000000000000 P a6315b057ded4e9cb09a53abc79dd676 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 22:57:25.744457 11710 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a6315b057ded4e9cb09a53abc79dd676 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:57:25.746520 11710 raft_consensus.cc:504] T 00000000000000000000000000000000 P a6315b057ded4e9cb09a53abc79dd676 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a6315b057ded4e9cb09a53abc79dd676" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 36003 } }
I20260809 22:57:25.746960 11710 leader_election.cc:302] T 00000000000000000000000000000000 P a6315b057ded4e9cb09a53abc79dd676 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a6315b057ded4e9cb09a53abc79dd676; no voters:
I20260809 22:57:25.747476 11710 leader_election.cc:288] T 00000000000000000000000000000000 P a6315b057ded4e9cb09a53abc79dd676 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 22:57:25.747601 11715 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a6315b057ded4e9cb09a53abc79dd676 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 22:57:25.748248 11715 raft_consensus.cc:686] T 00000000000000000000000000000000 P a6315b057ded4e9cb09a53abc79dd676 [term 1 LEADER]: Becoming Leader. State: Replica: a6315b057ded4e9cb09a53abc79dd676, State: Running, Role: LEADER
I20260809 22:57:25.748739 11715 consensus_queue.cc:237] T 00000000000000000000000000000000 P a6315b057ded4e9cb09a53abc79dd676 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a6315b057ded4e9cb09a53abc79dd676" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 36003 } }
I20260809 22:57:25.749351 11710 sys_catalog.cc:564] T 00000000000000000000000000000000 P a6315b057ded4e9cb09a53abc79dd676 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 22:57:25.751212 11718 sys_catalog.cc:455] T 00000000000000000000000000000000 P a6315b057ded4e9cb09a53abc79dd676 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a6315b057ded4e9cb09a53abc79dd676. Latest consensus state: current_term: 1 leader_uuid: "a6315b057ded4e9cb09a53abc79dd676" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a6315b057ded4e9cb09a53abc79dd676" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 36003 } } }
I20260809 22:57:25.751611 11718 sys_catalog.cc:458] T 00000000000000000000000000000000 P a6315b057ded4e9cb09a53abc79dd676 [sys.catalog]: This master's current role is: LEADER
I20260809 22:57:25.751497 11716 sys_catalog.cc:455] T 00000000000000000000000000000000 P a6315b057ded4e9cb09a53abc79dd676 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a6315b057ded4e9cb09a53abc79dd676" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a6315b057ded4e9cb09a53abc79dd676" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 36003 } } }
I20260809 22:57:25.751799 11716 sys_catalog.cc:458] T 00000000000000000000000000000000 P a6315b057ded4e9cb09a53abc79dd676 [sys.catalog]: This master's current role is: LEADER
I20260809 22:57:25.753790 11722 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 22:57:25.758610 11722 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 22:57:25.767251 11722 catalog_manager.cc:1347] Generated new cluster ID: 524d9e989ec24f55958d1d677ad097ed
I20260809 22:57:25.767349 11722 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 22:57:25.786000 11722 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 22:57:25.787197 11722 catalog_manager.cc:1504] Loading token signing keys...
I20260809 22:57:25.793390 11722 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a6315b057ded4e9cb09a53abc79dd676: Generated new TSK 0
I20260809 22:57:25.794097 11722 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.113.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.113.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.113.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.129.keytab.
Entry for principal kudu/127.15.113.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.129.keytab.
I20260809 22:57:25.878319 15814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskItYPJR/build/asan/bin/kudu
/tmp/dist-test-taskItYPJR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315935617919-15814-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315935617919-15814-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315935617919-15814-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315935617919-15814-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.113.129:0
--local_ip_for_outbound_sockets=127.15.113.129
--webserver_interface=127.15.113.129
--webserver_port=0
--tserver_master_addrs=127.15.113.190:36003
--keytab_file=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.129.keytab
--principal=kudu/127.15.113.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.113.148:46077
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315935617919-15814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315935617919-15814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315935617919-15814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:57:26.237903 11749 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:57:26.238188 11749 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:57:26.238325 11749 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:57:26.245398 11749 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:57:26.245504 11749 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:57:26.245648 11749 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.113.129
I20260809 22:57:26.255167 11749 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.113.148:46077
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315935617919-15814-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315935617919-15814-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.129.keytab
--principal=kudu/127.15.113.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.113.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315935617919-15814-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.113.129
--webserver_port=0
--tserver_master_addrs=127.15.113.190:36003
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.113.129
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315935617919-15814-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:57:26.256346 11749 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:57:26.257630 11749 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:57:26.269840 11759 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:57:26.269886 11758 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:57:26.269908 11762 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:57:26.270898 11749 server_base.cc:1034] running on GCE node
I20260809 22:57:26.271428 11749 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:57:26.272292 11749 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:57:26.273487 11749 hybrid_clock.cc:648] HybridClock initialized: now 1786316246273438 us; error 40 us; skew 500 ppm
I20260809 22:57:26.273788 11749 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:57:26 dist-test-slave-ncp9 krb5kdc[11600](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316246, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.113.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:57:26.276757 11749 init.cc:377] Logged in from keytab as kudu/127.15.113.129@KRBTEST.COM (short username kudu)
I20260809 22:57:26.278108 11749 webserver.cc:460] Webserver started at http://127.15.113.129:36565/ using document root <none> and password file <none>
I20260809 22:57:26.278745 11749 fs_manager.cc:359] Metadata directory not provided
I20260809 22:57:26.278864 11749 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:57:26.279155 11749 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:57:26.281987 11749 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315935617919-15814-0/minicluster-data/ts-0/data/instance:
uuid: "819262f3c90b4e1ea104e4c962bb79ae"
format_stamp: "Formatted at 2026-08-09 22:57:26 on dist-test-slave-ncp9"
server_key: "fb3e00d4989fed295744adb7623f0b3b"
server_key_iv: "2ceebc266d491f88826fc2f958450b41"
server_key_version: "encryptionkey@0"
I20260809 22:57:26.282742 11749 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315935617919-15814-0/minicluster-data/ts-0/wal/instance:
uuid: "819262f3c90b4e1ea104e4c962bb79ae"
format_stamp: "Formatted at 2026-08-09 22:57:26 on dist-test-slave-ncp9"
server_key: "fb3e00d4989fed295744adb7623f0b3b"
server_key_iv: "2ceebc266d491f88826fc2f958450b41"
server_key_version: "encryptionkey@0"
I20260809 22:57:26.287541 11749 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260809 22:57:26.290743 11775 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:57:26.291934 11749 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 22:57:26.292081 11749 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315935617919-15814-0/minicluster-data/ts-0/data,/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315935617919-15814-0/minicluster-data/ts-0/wal
uuid: "819262f3c90b4e1ea104e4c962bb79ae"
format_stamp: "Formatted at 2026-08-09 22:57:26 on dist-test-slave-ncp9"
server_key: "fb3e00d4989fed295744adb7623f0b3b"
server_key_iv: "2ceebc266d491f88826fc2f958450b41"
server_key_version: "encryptionkey@0"
I20260809 22:57:26.292268 11749 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315935617919-15814-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315935617919-15814-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315935617919-15814-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:57:26.308131 11749 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:57:26.311950 11749 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:57:26.312203 11749 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:57:26.313082 11749 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:57:26.314302 11749 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:57:26.314388 11749 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:57:26.314509 11749 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:57:26.314591 11749 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:57:26.343906 11749 rpc_server.cc:307] RPC server started. Bound to: 127.15.113.129:44053
I20260809 22:57:26.343947 11946 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.113.129:44053 every 8 connection(s)
I20260809 22:57:26.345188 11749 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315935617919-15814-0/minicluster-data/ts-0/data/info.pb
I20260809 22:57:26.350487 15814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskItYPJR/build/asan/bin/kudu as pid 11749
I20260809 22:57:26.350663 15814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315935617919-15814-0/minicluster-data/ts-0/wal/instance
I20260809 22:57:26.351166 15814 external_mini_cluster.cc:1428] Setting key d1142afeb2b5c7037d6e879d48152111
Aug 09 22:57:26 dist-test-slave-ncp9 krb5kdc[11600](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316246, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.113.129@KRBTEST.COM for kudu/127.15.113.190@KRBTEST.COM
I20260809 22:57:26.363834 11952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:57:26.347533 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.113.129:40655 (local address 127.15.113.190:36003)
0809 22:57:26.347977 (+ 444us) server_negotiation.cc:206] Beginning negotiation
0809 22:57:26.347982 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:57:26.348816 (+ 834us) server_negotiation.cc:407] Connection header received
0809 22:57:26.350645 (+ 1829us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:57:26.350652 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:57:26.350751 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:57:26.350856 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:57:26.352446 (+ 1590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:57:26.353303 (+ 857us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:57:26.354788 (+ 1485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:57:26.355189 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:57:26.355294 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:57:26.358597 (+ 3303us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:57:26.358602 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:57:26.358605 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:57:26.358637 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:57:26.361040 (+ 2403us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:57:26.361704 (+ 664us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:57:26.361710 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:57:26.361713 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:57:26.361832 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:57:26.362239 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:57:26.362244 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:57:26.362248 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:57:26.362503 (+ 255us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:57:26.362636 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0809 22:57:26.363280 (+ 644us) server_negotiation.cc:299] Negotiation successful
0809 22:57:26.363496 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":345,"thread_start_us":247,"threads_started":1}
I20260809 22:57:26.364727 11953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:57:26.347955 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:36003 (local address 127.15.113.129:40655)
0809 22:57:26.348644 (+ 689us) negotiation.cc:107] Waiting for socket to connect
0809 22:57:26.348692 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0809 22:57:26.350380 (+ 1688us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:57:26.351096 (+ 716us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:57:26.351104 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:57:26.351623 (+ 519us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:57:26.352280 (+ 657us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:57:26.352300 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:57:26.353458 (+ 1158us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:57:26.353462 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:57:26.354658 (+ 1196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:57:26.354670 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:57:26.355398 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:57:26.355405 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:57:26.355618 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:57:26.356255 (+ 637us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:57:26.356278 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:57:26.358392 (+ 2114us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:57:26.361162 (+ 2770us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:57:26.361171 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:57:26.361186 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:57:26.361593 (+ 407us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:57:26.361934 (+ 341us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:57:26.361939 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:57:26.361943 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:57:26.362123 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:57:26.362624 (+ 501us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:57:26.362631 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:57:26.362935 (+ 304us) client_negotiation.cc:769] Sending connection context
0809 22:57:26.363167 (+ 232us) client_negotiation.cc:240] Negotiation successful
0809 22:57:26.363658 (+ 491us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":462,"thread_start_us":267,"threads_started":1}
I20260809 22:57:26.366214 11948 heartbeater.cc:347] Connected to a master server at 127.15.113.190:36003
I20260809 22:57:26.366489 11948 heartbeater.cc:464] Registering TS with master...
I20260809 22:57:26.367167 11948 heartbeater.cc:511] Master 127.15.113.190:36003 requested a full tablet report, sending...
I20260809 22:57:26.369094 11660 ts_manager.cc:194] Registered new tserver with Master: 819262f3c90b4e1ea104e4c962bb79ae (127.15.113.129:44053)
I20260809 22:57:26.370510 11660 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.113.129@KRBTEST.COM'} at 127.15.113.129:40655
WARNING: no policy specified for kudu/127.15.113.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.113.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.113.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.130.keytab.
Entry for principal kudu/127.15.113.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.130.keytab.
I20260809 22:57:26.414217 15814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskItYPJR/build/asan/bin/kudu
/tmp/dist-test-taskItYPJR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315935617919-15814-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315935617919-15814-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315935617919-15814-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315935617919-15814-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.113.130:0
--local_ip_for_outbound_sockets=127.15.113.130
--webserver_interface=127.15.113.130
--webserver_port=0
--tserver_master_addrs=127.15.113.190:36003
--keytab_file=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.130.keytab
--principal=kudu/127.15.113.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.113.148:46077
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315935617919-15814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315935617919-15814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315935617919-15814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:57:26.742041 11958 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:57:26.742321 11958 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:57:26.742425 11958 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:57:26.749333 11958 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:57:26.749426 11958 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:57:26.749542 11958 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.113.130
I20260809 22:57:26.758679 11958 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.113.148:46077
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315935617919-15814-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315935617919-15814-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.130.keytab
--principal=kudu/127.15.113.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.113.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315935617919-15814-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.113.130
--webserver_port=0
--tserver_master_addrs=127.15.113.190:36003
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.113.130
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315935617919-15814-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:57:26.759819 11958 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:57:26.761065 11958 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:57:26.773458 11964 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:57:26.773504 11965 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:57:26.773527 11969 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:57:26.774458 11958 server_base.cc:1034] running on GCE node
I20260809 22:57:26.774962 11958 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:57:26.775784 11958 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:57:26.776979 11958 hybrid_clock.cc:648] HybridClock initialized: now 1786316246776935 us; error 32 us; skew 500 ppm
I20260809 22:57:26.777266 11958 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:57:26 dist-test-slave-ncp9 krb5kdc[11600](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316246, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.113.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:57:26.780100 11958 init.cc:377] Logged in from keytab as kudu/127.15.113.130@KRBTEST.COM (short username kudu)
I20260809 22:57:26.781446 11958 webserver.cc:460] Webserver started at http://127.15.113.130:32869/ using document root <none> and password file <none>
I20260809 22:57:26.782053 11958 fs_manager.cc:359] Metadata directory not provided
I20260809 22:57:26.782153 11958 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:57:26.782423 11958 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:57:26.785197 11958 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315935617919-15814-0/minicluster-data/ts-1/data/instance:
uuid: "0d4ded9d98ea469f8318c3e3fec0a291"
format_stamp: "Formatted at 2026-08-09 22:57:26 on dist-test-slave-ncp9"
server_key: "fa077faf738378b5d5aa72770cbb89f8"
server_key_iv: "4e713643fe5b96e5b91dc79c17468f8e"
server_key_version: "encryptionkey@0"
I20260809 22:57:26.785904 11958 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315935617919-15814-0/minicluster-data/ts-1/wal/instance:
uuid: "0d4ded9d98ea469f8318c3e3fec0a291"
format_stamp: "Formatted at 2026-08-09 22:57:26 on dist-test-slave-ncp9"
server_key: "fa077faf738378b5d5aa72770cbb89f8"
server_key_iv: "4e713643fe5b96e5b91dc79c17468f8e"
server_key_version: "encryptionkey@0"
I20260809 22:57:26.790740 11958 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.003s
I20260809 22:57:26.794019 11981 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:57:26.795272 11958 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 22:57:26.795423 11958 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315935617919-15814-0/minicluster-data/ts-1/data,/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315935617919-15814-0/minicluster-data/ts-1/wal
uuid: "0d4ded9d98ea469f8318c3e3fec0a291"
format_stamp: "Formatted at 2026-08-09 22:57:26 on dist-test-slave-ncp9"
server_key: "fa077faf738378b5d5aa72770cbb89f8"
server_key_iv: "4e713643fe5b96e5b91dc79c17468f8e"
server_key_version: "encryptionkey@0"
I20260809 22:57:26.795600 11958 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315935617919-15814-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315935617919-15814-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315935617919-15814-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:57:26.808264 11958 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:57:26.811718 11958 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:57:26.811959 11958 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:57:26.812781 11958 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:57:26.814038 11958 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:57:26.814106 11958 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:57:26.814201 11958 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:57:26.814251 11958 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:57:26.844304 11958 rpc_server.cc:307] RPC server started. Bound to: 127.15.113.130:39795
I20260809 22:57:26.844362 12138 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.113.130:39795 every 8 connection(s)
I20260809 22:57:26.845556 11958 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315935617919-15814-0/minicluster-data/ts-1/data/info.pb
I20260809 22:57:26.850452 15814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskItYPJR/build/asan/bin/kudu as pid 11958
I20260809 22:57:26.850667 15814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315935617919-15814-0/minicluster-data/ts-1/wal/instance
I20260809 22:57:26.851233 15814 external_mini_cluster.cc:1428] Setting key d02d558559a9529fff80585d2691a3d2
Aug 09 22:57:26 dist-test-slave-ncp9 krb5kdc[11600](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316246, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.113.130@KRBTEST.COM for kudu/127.15.113.190@KRBTEST.COM
I20260809 22:57:26.866032 11952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:57:26.848009 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.113.130:59501 (local address 127.15.113.190:36003)
0809 22:57:26.848175 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0809 22:57:26.848179 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 22:57:26.849426 (+ 1247us) server_negotiation.cc:407] Connection header received
0809 22:57:26.851759 (+ 2333us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:57:26.851765 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:57:26.851851 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:57:26.851968 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:57:26.853678 (+ 1710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:57:26.854372 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:57:26.856536 (+ 2164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:57:26.856998 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:57:26.857085 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:57:26.860706 (+ 3621us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:57:26.860711 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:57:26.860714 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:57:26.860746 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:57:26.863186 (+ 2440us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:57:26.863843 (+ 657us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:57:26.863847 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:57:26.863850 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:57:26.863980 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:57:26.864398 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:57:26.864402 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:57:26.864404 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:57:26.864659 (+ 255us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:57:26.864787 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0809 22:57:26.865477 (+ 690us) server_negotiation.cc:299] Negotiation successful
0809 22:57:26.865705 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260809 22:57:26.867127 12142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:57:26.848461 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:36003 (local address 127.15.113.130:59501)
0809 22:57:26.849248 (+ 787us) negotiation.cc:107] Waiting for socket to connect
0809 22:57:26.849290 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0809 22:57:26.851459 (+ 2169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:57:26.852207 (+ 748us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:57:26.852218 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:57:26.852749 (+ 531us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:57:26.853514 (+ 765us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:57:26.853540 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:57:26.854525 (+ 985us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:57:26.854530 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:57:26.856392 (+ 1862us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:57:26.856407 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:57:26.857227 (+ 820us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:57:26.857234 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:57:26.857513 (+ 279us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:57:26.858158 (+ 645us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:57:26.858184 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:57:26.860494 (+ 2310us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:57:26.863321 (+ 2827us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:57:26.863330 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:57:26.863345 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:57:26.863734 (+ 389us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:57:26.864083 (+ 349us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:57:26.864088 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:57:26.864092 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:57:26.864292 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:57:26.864779 (+ 487us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:57:26.864786 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:57:26.865130 (+ 344us) client_negotiation.cc:769] Sending connection context
0809 22:57:26.865368 (+ 238us) client_negotiation.cc:240] Negotiation successful
0809 22:57:26.866204 (+ 836us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":506,"thread_start_us":267,"threads_started":1}
I20260809 22:57:26.868487 12140 heartbeater.cc:347] Connected to a master server at 127.15.113.190:36003
I20260809 22:57:26.868824 12140 heartbeater.cc:464] Registering TS with master...
I20260809 22:57:26.869660 12140 heartbeater.cc:511] Master 127.15.113.190:36003 requested a full tablet report, sending...
I20260809 22:57:26.871088 11660 ts_manager.cc:194] Registered new tserver with Master: 0d4ded9d98ea469f8318c3e3fec0a291 (127.15.113.130:39795)
I20260809 22:57:26.871847 11660 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.113.130@KRBTEST.COM'} at 127.15.113.130:59501
WARNING: no policy specified for kudu/127.15.113.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.113.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.113.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.131.keytab.
Entry for principal kudu/127.15.113.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.131.keytab.
I20260809 22:57:26.919837 15814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskItYPJR/build/asan/bin/kudu
/tmp/dist-test-taskItYPJR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315935617919-15814-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315935617919-15814-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315935617919-15814-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315935617919-15814-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.113.131:0
--local_ip_for_outbound_sockets=127.15.113.131
--webserver_interface=127.15.113.131
--webserver_port=0
--tserver_master_addrs=127.15.113.190:36003
--keytab_file=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.131.keytab
--principal=kudu/127.15.113.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.113.148:46077
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315935617919-15814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315935617919-15814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315935617919-15814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:57:27.251581 12147 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:57:27.251837 12147 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:57:27.251935 12147 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:57:27.258841 12147 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:57:27.258934 12147 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:57:27.259045 12147 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.113.131
I20260809 22:57:27.268040 12147 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.113.148:46077
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315935617919-15814-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315935617919-15814-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.131.keytab
--principal=kudu/127.15.113.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.113.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315935617919-15814-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.113.131
--webserver_port=0
--tserver_master_addrs=127.15.113.190:36003
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.113.131
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315935617919-15814-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:57:27.269189 12147 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:57:27.270371 12147 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:57:27.282658 12156 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:57:27.282694 12160 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:57:27.282577 12157 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:57:27.283488 12147 server_base.cc:1034] running on GCE node
I20260809 22:57:27.284045 12147 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:57:27.284914 12147 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:57:27.286123 12147 hybrid_clock.cc:648] HybridClock initialized: now 1786316247286066 us; error 50 us; skew 500 ppm
I20260809 22:57:27.286398 12147 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:57:27 dist-test-slave-ncp9 krb5kdc[11600](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316247, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.113.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:57:27.289250 12147 init.cc:377] Logged in from keytab as kudu/127.15.113.131@KRBTEST.COM (short username kudu)
I20260809 22:57:27.290573 12147 webserver.cc:460] Webserver started at http://127.15.113.131:38653/ using document root <none> and password file <none>
I20260809 22:57:27.291177 12147 fs_manager.cc:359] Metadata directory not provided
I20260809 22:57:27.291293 12147 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:57:27.291590 12147 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:57:27.294473 12147 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315935617919-15814-0/minicluster-data/ts-2/data/instance:
uuid: "6cda25ce8218430ea12a8a22df0c9d63"
format_stamp: "Formatted at 2026-08-09 22:57:27 on dist-test-slave-ncp9"
server_key: "c685ac2c01d83d84c95ae8372c23dd1e"
server_key_iv: "b8c716829b692d502c76a9b60e027e00"
server_key_version: "encryptionkey@0"
I20260809 22:57:27.295176 12147 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315935617919-15814-0/minicluster-data/ts-2/wal/instance:
uuid: "6cda25ce8218430ea12a8a22df0c9d63"
format_stamp: "Formatted at 2026-08-09 22:57:27 on dist-test-slave-ncp9"
server_key: "c685ac2c01d83d84c95ae8372c23dd1e"
server_key_iv: "b8c716829b692d502c76a9b60e027e00"
server_key_version: "encryptionkey@0"
I20260809 22:57:27.300048 12147 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.004s
I20260809 22:57:27.303273 12168 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:57:27.304432 12147 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 22:57:27.304601 12147 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315935617919-15814-0/minicluster-data/ts-2/data,/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315935617919-15814-0/minicluster-data/ts-2/wal
uuid: "6cda25ce8218430ea12a8a22df0c9d63"
format_stamp: "Formatted at 2026-08-09 22:57:27 on dist-test-slave-ncp9"
server_key: "c685ac2c01d83d84c95ae8372c23dd1e"
server_key_iv: "b8c716829b692d502c76a9b60e027e00"
server_key_version: "encryptionkey@0"
I20260809 22:57:27.304772 12147 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315935617919-15814-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315935617919-15814-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315935617919-15814-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:57:27.322571 12147 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:57:27.325968 12147 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:57:27.326222 12147 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:57:27.327103 12147 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:57:27.328317 12147 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:57:27.328393 12147 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:57:27.328521 12147 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:57:27.328573 12147 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:57:27.358484 12147 rpc_server.cc:307] RPC server started. Bound to: 127.15.113.131:32893
I20260809 22:57:27.358527 12324 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.113.131:32893 every 8 connection(s)
I20260809 22:57:27.359615 12147 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315935617919-15814-0/minicluster-data/ts-2/data/info.pb
I20260809 22:57:27.366075 15814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskItYPJR/build/asan/bin/kudu as pid 12147
I20260809 22:57:27.366285 15814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315935617919-15814-0/minicluster-data/ts-2/wal/instance
I20260809 22:57:27.366834 15814 external_mini_cluster.cc:1428] Setting key ecaf86062bf217aee370c21d0609f734
Aug 09 22:57:27 dist-test-slave-ncp9 krb5kdc[11600](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316247, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.113.131@KRBTEST.COM for kudu/127.15.113.190@KRBTEST.COM
I20260809 22:57:27.373099 11948 heartbeater.cc:503] Master 127.15.113.190:36003 was elected leader, sending a full tablet report...
I20260809 22:57:27.378767 11952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:57:27.361954 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.113.131:33695 (local address 127.15.113.190:36003)
0809 22:57:27.362125 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0809 22:57:27.362131 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:57:27.363185 (+ 1054us) server_negotiation.cc:407] Connection header received
0809 22:57:27.364880 (+ 1695us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:57:27.364885 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:57:27.365010 (+ 125us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:57:27.365097 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:57:27.366796 (+ 1699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:57:27.367769 (+ 973us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:57:27.369302 (+ 1533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:57:27.369703 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:57:27.369806 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:57:27.372943 (+ 3137us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:57:27.372950 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:57:27.372955 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:57:27.372986 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:57:27.375816 (+ 2830us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:57:27.376610 (+ 794us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:57:27.376615 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:57:27.376619 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:57:27.376765 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:57:27.377202 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:57:27.377207 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:57:27.377210 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:57:27.377467 (+ 257us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:57:27.377603 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0809 22:57:27.378239 (+ 636us) server_negotiation.cc:299] Negotiation successful
0809 22:57:27.378445 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260809 22:57:27.379560 12330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:57:27.362322 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:36003 (local address 127.15.113.131:33695)
0809 22:57:27.363019 (+ 697us) negotiation.cc:107] Waiting for socket to connect
0809 22:57:27.363055 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0809 22:57:27.364646 (+ 1591us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:57:27.365290 (+ 644us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:57:27.365297 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:57:27.365862 (+ 565us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:57:27.366617 (+ 755us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:57:27.366656 (+ 39us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:57:27.367923 (+ 1267us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:57:27.367927 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:57:27.369169 (+ 1242us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:57:27.369181 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:57:27.369913 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:57:27.369921 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:57:27.370124 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:57:27.370706 (+ 582us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:57:27.370726 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:57:27.372733 (+ 2007us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:57:27.375961 (+ 3228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:57:27.375970 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:57:27.375984 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:57:27.376508 (+ 524us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:57:27.376872 (+ 364us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:57:27.376878 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:57:27.376881 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:57:27.377093 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:57:27.377588 (+ 495us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:57:27.377595 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:57:27.377894 (+ 299us) client_negotiation.cc:769] Sending connection context
0809 22:57:27.378119 (+ 225us) client_negotiation.cc:240] Negotiation successful
0809 22:57:27.378569 (+ 450us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":468,"thread_start_us":277,"threads_started":1}
I20260809 22:57:27.380827 12325 heartbeater.cc:347] Connected to a master server at 127.15.113.190:36003
I20260809 22:57:27.381101 12325 heartbeater.cc:464] Registering TS with master...
I20260809 22:57:27.381834 12325 heartbeater.cc:511] Master 127.15.113.190:36003 requested a full tablet report, sending...
I20260809 22:57:27.383064 11659 ts_manager.cc:194] Registered new tserver with Master: 6cda25ce8218430ea12a8a22df0c9d63 (127.15.113.131:32893)
I20260809 22:57:27.383805 11659 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.113.131@KRBTEST.COM'} at 127.15.113.131:33695
I20260809 22:57:27.393572 15814 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 22:57:27.409490 11952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:57:27.397649 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45484 (local address 127.15.113.190:36003)
0809 22:57:27.397808 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0809 22:57:27.397812 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 22:57:27.398197 (+ 385us) server_negotiation.cc:407] Connection header received
0809 22:57:27.398325 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:57:27.398329 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:57:27.398384 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:57:27.398456 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:57:27.399927 (+ 1471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:57:27.400616 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:57:27.401861 (+ 1245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:57:27.402247 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:57:27.402331 (+ 84us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:57:27.404692 (+ 2361us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:57:27.404697 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:57:27.404700 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:57:27.404727 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:57:27.407071 (+ 2344us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:57:27.407808 (+ 737us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:57:27.407812 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:57:27.407814 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:57:27.407926 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:57:27.408268 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:57:27.408271 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:57:27.408273 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:57:27.408605 (+ 332us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:57:27.408792 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0809 22:57:27.409166 (+ 374us) server_negotiation.cc:299] Negotiation successful
0809 22:57:27.409287 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260809 22:57:27.412949 11659 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45484:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 22:57:27.419464 11659 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 22:57:27.453898 12351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:57:27.431245 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38964 (local address 127.15.113.129:44053)
0809 22:57:27.442102 (+ 10857us) server_negotiation.cc:206] Beginning negotiation
0809 22:57:27.442112 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 22:57:27.443754 (+ 1642us) server_negotiation.cc:407] Connection header received
0809 22:57:27.443870 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:57:27.443880 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:57:27.444157 (+ 277us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:57:27.445115 (+ 958us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:57:27.445385 (+ 270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:57:27.448064 (+ 2679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:57:27.450793 (+ 2729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:57:27.452310 (+ 1517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:57:27.452499 (+ 189us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:57:27.452693 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0809 22:57:27.453393 (+ 700us) server_negotiation.cc:299] Negotiation successful
0809 22:57:27.453561 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10674,"thread_start_us":10503,"threads_started":1}
I20260809 22:57:27.454494 12348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:57:27.430998 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.129:44053 (local address 127.0.0.1:38964)
0809 22:57:27.443463 (+ 12465us) negotiation.cc:107] Waiting for socket to connect
0809 22:57:27.443483 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 22:57:27.443608 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:57:27.444418 (+ 810us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:57:27.444426 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:57:27.444461 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:57:27.444877 (+ 416us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:57:27.444911 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:57:27.448231 (+ 3320us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:57:27.448237 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:57:27.450524 (+ 2287us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:57:27.450585 (+ 61us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:57:27.452504 (+ 1919us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:57:27.452510 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:57:27.452694 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:57:27.452745 (+ 51us) client_negotiation.cc:769] Sending connection context
0809 22:57:27.452864 (+ 119us) client_negotiation.cc:240] Negotiation successful
0809 22:57:27.453073 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12330,"threads_started":1}
I20260809 22:57:27.458779 11842 tablet_service.cc:1467] Processing CreateTablet for tablet 6c2562fac19b45e1a3c542433ae1b372 (DEFAULT_TABLE table=test-table [id=ea33194facc34a0fa181786d814542f2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:57:27.460060 11842 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6c2562fac19b45e1a3c542433ae1b372. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:57:27.462648 12352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:57:27.430585 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.130:39795 (local address 127.0.0.1:41154)
0809 22:57:27.439288 (+ 8703us) negotiation.cc:107] Waiting for socket to connect
0809 22:57:27.439345 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0809 22:57:27.439523 (+ 178us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:57:27.455962 (+ 16439us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:57:27.455967 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:57:27.455996 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:57:27.456406 (+ 410us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:57:27.456415 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:57:27.457762 (+ 1347us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:57:27.457765 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:57:27.459890 (+ 2125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:57:27.459910 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:57:27.461783 (+ 1873us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:57:27.461789 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:57:27.461962 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:57:27.461994 (+ 32us) client_negotiation.cc:769] Sending connection context
0809 22:57:27.462100 (+ 106us) client_negotiation.cc:240] Negotiation successful
0809 22:57:27.462245 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8409,"mutex_wait_us":83,"thread_start_us":17000,"threads_started":1}
I20260809 22:57:27.462674 12350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:57:27.431027 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41154 (local address 127.15.113.130:39795)
0809 22:57:27.455050 (+ 24023us) server_negotiation.cc:206] Beginning negotiation
0809 22:57:27.455056 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:57:27.455102 (+ 46us) server_negotiation.cc:407] Connection header received
0809 22:57:27.455201 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:57:27.455208 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:57:27.455464 (+ 256us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:57:27.455671 (+ 207us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:57:27.456553 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:57:27.457618 (+ 1065us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:57:27.460104 (+ 2486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:57:27.461570 (+ 1466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:57:27.461733 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:57:27.461913 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0809 22:57:27.462166 (+ 253us) server_negotiation.cc:299] Negotiation successful
0809 22:57:27.462353 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23886,"thread_start_us":23773,"threads_started":1}
I20260809 22:57:27.463217 12355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:57:27.433151 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44720 (local address 127.15.113.131:32893)
0809 22:57:27.454306 (+ 21155us) server_negotiation.cc:206] Beginning negotiation
0809 22:57:27.454315 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 22:57:27.454869 (+ 554us) server_negotiation.cc:407] Connection header received
0809 22:57:27.455258 (+ 389us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:57:27.455265 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:57:27.455464 (+ 199us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:57:27.455664 (+ 200us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:57:27.456274 (+ 610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:57:27.457822 (+ 1548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:57:27.460188 (+ 2366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:57:27.461562 (+ 1374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:57:27.461765 (+ 203us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:57:27.461913 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0809 22:57:27.462745 (+ 832us) server_negotiation.cc:299] Negotiation successful
0809 22:57:27.462902 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20970,"thread_start_us":20858,"threads_started":1}
I20260809 22:57:27.463693 12349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:57:27.432546 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.131:32893 (local address 127.0.0.1:44720)
0809 22:57:27.454748 (+ 22202us) negotiation.cc:107] Waiting for socket to connect
0809 22:57:27.454767 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 22:57:27.455048 (+ 281us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:57:27.455677 (+ 629us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:57:27.455682 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:57:27.455707 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:57:27.456050 (+ 343us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:57:27.456065 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:57:27.457966 (+ 1901us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:57:27.457971 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:57:27.460027 (+ 2056us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:57:27.460038 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:57:27.462060 (+ 2022us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:57:27.462065 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:57:27.462236 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:57:27.462268 (+ 32us) client_negotiation.cc:769] Sending connection context
0809 22:57:27.462357 (+ 89us) client_negotiation.cc:240] Negotiation successful
0809 22:57:27.462475 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":22084,"mutex_wait_us":2,"thread_start_us":6316,"threads_started":1}
I20260809 22:57:27.466378 12051 tablet_service.cc:1467] Processing CreateTablet for tablet 6c2562fac19b45e1a3c542433ae1b372 (DEFAULT_TABLE table=test-table [id=ea33194facc34a0fa181786d814542f2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:57:27.467794 12051 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6c2562fac19b45e1a3c542433ae1b372. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:57:27.467415 12234 tablet_service.cc:1467] Processing CreateTablet for tablet 6c2562fac19b45e1a3c542433ae1b372 (DEFAULT_TABLE table=test-table [id=ea33194facc34a0fa181786d814542f2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:57:27.468835 12234 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6c2562fac19b45e1a3c542433ae1b372. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:57:27.472923 12356 tablet_bootstrap.cc:492] T 6c2562fac19b45e1a3c542433ae1b372 P 819262f3c90b4e1ea104e4c962bb79ae: Bootstrap starting.
I20260809 22:57:27.475829 12356 tablet_bootstrap.cc:654] T 6c2562fac19b45e1a3c542433ae1b372 P 819262f3c90b4e1ea104e4c962bb79ae: Neither blocks nor log segments found. Creating new log.
I20260809 22:57:27.476871 12356 log.cc:824] T 6c2562fac19b45e1a3c542433ae1b372 P 819262f3c90b4e1ea104e4c962bb79ae: Log is configured to *not* fsync() on all Append() calls
I20260809 22:57:27.479563 12356 tablet_bootstrap.cc:492] T 6c2562fac19b45e1a3c542433ae1b372 P 819262f3c90b4e1ea104e4c962bb79ae: No bootstrap required, opened a new log
I20260809 22:57:27.479909 12356 ts_tablet_manager.cc:1397] T 6c2562fac19b45e1a3c542433ae1b372 P 819262f3c90b4e1ea104e4c962bb79ae: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.007s
I20260809 22:57:27.481997 12358 tablet_bootstrap.cc:492] T 6c2562fac19b45e1a3c542433ae1b372 P 0d4ded9d98ea469f8318c3e3fec0a291: Bootstrap starting.
I20260809 22:57:27.483444 12359 tablet_bootstrap.cc:492] T 6c2562fac19b45e1a3c542433ae1b372 P 6cda25ce8218430ea12a8a22df0c9d63: Bootstrap starting.
I20260809 22:57:27.485415 12358 tablet_bootstrap.cc:654] T 6c2562fac19b45e1a3c542433ae1b372 P 0d4ded9d98ea469f8318c3e3fec0a291: Neither blocks nor log segments found. Creating new log.
I20260809 22:57:27.486145 12359 tablet_bootstrap.cc:654] T 6c2562fac19b45e1a3c542433ae1b372 P 6cda25ce8218430ea12a8a22df0c9d63: Neither blocks nor log segments found. Creating new log.
I20260809 22:57:27.486584 12358 log.cc:824] T 6c2562fac19b45e1a3c542433ae1b372 P 0d4ded9d98ea469f8318c3e3fec0a291: Log is configured to *not* fsync() on all Append() calls
I20260809 22:57:27.487046 12359 log.cc:824] T 6c2562fac19b45e1a3c542433ae1b372 P 6cda25ce8218430ea12a8a22df0c9d63: Log is configured to *not* fsync() on all Append() calls
I20260809 22:57:27.489463 12358 tablet_bootstrap.cc:492] T 6c2562fac19b45e1a3c542433ae1b372 P 0d4ded9d98ea469f8318c3e3fec0a291: No bootstrap required, opened a new log
I20260809 22:57:27.489274 12356 raft_consensus.cc:348] T 6c2562fac19b45e1a3c542433ae1b372 P 819262f3c90b4e1ea104e4c962bb79ae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d4ded9d98ea469f8318c3e3fec0a291" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 39795 } } peers { permanent_uuid: "819262f3c90b4e1ea104e4c962bb79ae" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 44053 } } peers { permanent_uuid: "6cda25ce8218430ea12a8a22df0c9d63" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 32893 } }
I20260809 22:57:27.489591 12356 raft_consensus.cc:374] T 6c2562fac19b45e1a3c542433ae1b372 P 819262f3c90b4e1ea104e4c962bb79ae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:57:27.489686 12356 raft_consensus.cc:729] T 6c2562fac19b45e1a3c542433ae1b372 P 819262f3c90b4e1ea104e4c962bb79ae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 819262f3c90b4e1ea104e4c962bb79ae, State: Initialized, Role: FOLLOWER
I20260809 22:57:27.489732 12358 ts_tablet_manager.cc:1397] T 6c2562fac19b45e1a3c542433ae1b372 P 0d4ded9d98ea469f8318c3e3fec0a291: Time spent bootstrapping tablet: real 0.008s user 0.007s sys 0.000s
I20260809 22:57:27.490006 12359 tablet_bootstrap.cc:492] T 6c2562fac19b45e1a3c542433ae1b372 P 6cda25ce8218430ea12a8a22df0c9d63: No bootstrap required, opened a new log
I20260809 22:57:27.490264 12359 ts_tablet_manager.cc:1397] T 6c2562fac19b45e1a3c542433ae1b372 P 6cda25ce8218430ea12a8a22df0c9d63: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260809 22:57:27.490238 12356 consensus_queue.cc:260] T 6c2562fac19b45e1a3c542433ae1b372 P 819262f3c90b4e1ea104e4c962bb79ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d4ded9d98ea469f8318c3e3fec0a291" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 39795 } } peers { permanent_uuid: "819262f3c90b4e1ea104e4c962bb79ae" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 44053 } } peers { permanent_uuid: "6cda25ce8218430ea12a8a22df0c9d63" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 32893 } }
I20260809 22:57:27.491847 12356 ts_tablet_manager.cc:1428] T 6c2562fac19b45e1a3c542433ae1b372 P 819262f3c90b4e1ea104e4c962bb79ae: Time spent starting tablet: real 0.012s user 0.010s sys 0.000s
I20260809 22:57:27.500090 12358 raft_consensus.cc:348] T 6c2562fac19b45e1a3c542433ae1b372 P 0d4ded9d98ea469f8318c3e3fec0a291 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d4ded9d98ea469f8318c3e3fec0a291" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 39795 } } peers { permanent_uuid: "819262f3c90b4e1ea104e4c962bb79ae" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 44053 } } peers { permanent_uuid: "6cda25ce8218430ea12a8a22df0c9d63" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 32893 } }
I20260809 22:57:27.500227 12359 raft_consensus.cc:348] T 6c2562fac19b45e1a3c542433ae1b372 P 6cda25ce8218430ea12a8a22df0c9d63 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d4ded9d98ea469f8318c3e3fec0a291" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 39795 } } peers { permanent_uuid: "819262f3c90b4e1ea104e4c962bb79ae" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 44053 } } peers { permanent_uuid: "6cda25ce8218430ea12a8a22df0c9d63" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 32893 } }
I20260809 22:57:27.500568 12358 raft_consensus.cc:374] T 6c2562fac19b45e1a3c542433ae1b372 P 0d4ded9d98ea469f8318c3e3fec0a291 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:57:27.500586 12359 raft_consensus.cc:374] T 6c2562fac19b45e1a3c542433ae1b372 P 6cda25ce8218430ea12a8a22df0c9d63 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:57:27.500676 12359 raft_consensus.cc:729] T 6c2562fac19b45e1a3c542433ae1b372 P 6cda25ce8218430ea12a8a22df0c9d63 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6cda25ce8218430ea12a8a22df0c9d63, State: Initialized, Role: FOLLOWER
I20260809 22:57:27.500676 12358 raft_consensus.cc:729] T 6c2562fac19b45e1a3c542433ae1b372 P 0d4ded9d98ea469f8318c3e3fec0a291 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d4ded9d98ea469f8318c3e3fec0a291, State: Initialized, Role: FOLLOWER
I20260809 22:57:27.501297 12359 consensus_queue.cc:260] T 6c2562fac19b45e1a3c542433ae1b372 P 6cda25ce8218430ea12a8a22df0c9d63 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d4ded9d98ea469f8318c3e3fec0a291" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 39795 } } peers { permanent_uuid: "819262f3c90b4e1ea104e4c962bb79ae" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 44053 } } peers { permanent_uuid: "6cda25ce8218430ea12a8a22df0c9d63" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 32893 } }
I20260809 22:57:27.501298 12358 consensus_queue.cc:260] T 6c2562fac19b45e1a3c542433ae1b372 P 0d4ded9d98ea469f8318c3e3fec0a291 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d4ded9d98ea469f8318c3e3fec0a291" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 39795 } } peers { permanent_uuid: "819262f3c90b4e1ea104e4c962bb79ae" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 44053 } } peers { permanent_uuid: "6cda25ce8218430ea12a8a22df0c9d63" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 32893 } }
I20260809 22:57:27.502357 12140 heartbeater.cc:503] Master 127.15.113.190:36003 was elected leader, sending a full tablet report...
I20260809 22:57:27.502794 12325 heartbeater.cc:503] Master 127.15.113.190:36003 was elected leader, sending a full tablet report...
I20260809 22:57:27.503141 12358 ts_tablet_manager.cc:1428] T 6c2562fac19b45e1a3c542433ae1b372 P 0d4ded9d98ea469f8318c3e3fec0a291: Time spent starting tablet: real 0.013s user 0.012s sys 0.000s
I20260809 22:57:27.503407 12359 ts_tablet_manager.cc:1428] T 6c2562fac19b45e1a3c542433ae1b372 P 6cda25ce8218430ea12a8a22df0c9d63: Time spent starting tablet: real 0.013s user 0.013s sys 0.000s
W20260809 22:57:27.597422 12141 tablet.cc:2369] T 6c2562fac19b45e1a3c542433ae1b372 P 0d4ded9d98ea469f8318c3e3fec0a291: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 22:57:27.597420 11951 tablet.cc:2369] T 6c2562fac19b45e1a3c542433ae1b372 P 819262f3c90b4e1ea104e4c962bb79ae: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 22:57:27.604645 12366 raft_consensus.cc:482] T 6c2562fac19b45e1a3c542433ae1b372 P 0d4ded9d98ea469f8318c3e3fec0a291 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 22:57:27.604934 12366 raft_consensus.cc:504] T 6c2562fac19b45e1a3c542433ae1b372 P 0d4ded9d98ea469f8318c3e3fec0a291 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d4ded9d98ea469f8318c3e3fec0a291" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 39795 } } peers { permanent_uuid: "819262f3c90b4e1ea104e4c962bb79ae" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 44053 } } peers { permanent_uuid: "6cda25ce8218430ea12a8a22df0c9d63" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 32893 } }
I20260809 22:57:27.606622 12366 leader_election.cc:288] T 6c2562fac19b45e1a3c542433ae1b372 P 0d4ded9d98ea469f8318c3e3fec0a291 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 819262f3c90b4e1ea104e4c962bb79ae (127.15.113.129:44053), 6cda25ce8218430ea12a8a22df0c9d63 (127.15.113.131:32893)
W20260809 22:57:27.611042 12329 tablet.cc:2369] T 6c2562fac19b45e1a3c542433ae1b372 P 6cda25ce8218430ea12a8a22df0c9d63: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 22:57:27.615077 12351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:57:27.607031 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.113.130:50313 (local address 127.15.113.129:44053)
0809 22:57:27.607427 (+ 396us) server_negotiation.cc:206] Beginning negotiation
0809 22:57:27.607434 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:57:27.607957 (+ 523us) server_negotiation.cc:407] Connection header received
0809 22:57:27.608151 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:57:27.608183 (+ 32us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:57:27.608266 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:57:27.608363 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:57:27.609057 (+ 694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:57:27.610139 (+ 1082us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:57:27.612864 (+ 2725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:57:27.614072 (+ 1208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:57:27.614229 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:57:27.614303 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0809 22:57:27.614708 (+ 405us) server_negotiation.cc:299] Negotiation successful
0809 22:57:27.614834 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":234}
I20260809 22:57:27.615072 12355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:57:27.607039 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.113.130:51831 (local address 127.15.113.131:32893)
0809 22:57:27.607288 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0809 22:57:27.607296 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 22:57:27.608186 (+ 890us) server_negotiation.cc:407] Connection header received
0809 22:57:27.608342 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:57:27.608347 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:57:27.608415 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:57:27.608491 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:57:27.609200 (+ 709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:57:27.610275 (+ 1075us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:57:27.612861 (+ 2586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:57:27.614070 (+ 1209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:57:27.614242 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:57:27.614303 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0809 22:57:27.614708 (+ 405us) server_negotiation.cc:299] Negotiation successful
0809 22:57:27.614834 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260809 22:57:27.615314 12369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:57:27.606823 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.129:44053 (local address 127.15.113.130:50313)
0809 22:57:27.607838 (+ 1015us) negotiation.cc:107] Waiting for socket to connect
0809 22:57:27.607862 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0809 22:57:27.608024 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:57:27.608399 (+ 375us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:57:27.608404 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:57:27.608432 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:57:27.608884 (+ 452us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:57:27.608927 (+ 43us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:57:27.610255 (+ 1328us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:57:27.610259 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:57:27.612707 (+ 2448us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:57:27.612717 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:57:27.614317 (+ 1600us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:57:27.614324 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:57:27.614512 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:57:27.614579 (+ 67us) client_negotiation.cc:769] Sending connection context
0809 22:57:27.614678 (+ 99us) client_negotiation.cc:240] Negotiation successful
0809 22:57:27.614842 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":882,"mutex_wait_us":104,"thread_start_us":944,"threads_started":1}
I20260809 22:57:27.615343 12367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:57:27.607076 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.131:32893 (local address 127.15.113.130:51831)
0809 22:57:27.608097 (+ 1021us) negotiation.cc:107] Waiting for socket to connect
0809 22:57:27.608114 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 22:57:27.608226 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:57:27.608546 (+ 320us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:57:27.608550 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:57:27.608589 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:57:27.609026 (+ 437us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:57:27.609037 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:57:27.610388 (+ 1351us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:57:27.610393 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:57:27.612698 (+ 2305us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:57:27.612714 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:57:27.614324 (+ 1610us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:57:27.614330 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:57:27.614514 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:57:27.614587 (+ 73us) client_negotiation.cc:769] Sending connection context
0809 22:57:27.614678 (+ 91us) client_negotiation.cc:240] Negotiation successful
0809 22:57:27.614842 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":882,"mutex_wait_us":124,"thread_start_us":441,"threads_started":1}
I20260809 22:57:27.616328 12264 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6c2562fac19b45e1a3c542433ae1b372" candidate_uuid: "0d4ded9d98ea469f8318c3e3fec0a291" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6cda25ce8218430ea12a8a22df0c9d63" is_pre_election: true
I20260809 22:57:27.616326 11879 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6c2562fac19b45e1a3c542433ae1b372" candidate_uuid: "0d4ded9d98ea469f8318c3e3fec0a291" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "819262f3c90b4e1ea104e4c962bb79ae" is_pre_election: true
I20260809 22:57:27.616938 11879 raft_consensus.cc:2454] T 6c2562fac19b45e1a3c542433ae1b372 P 819262f3c90b4e1ea104e4c962bb79ae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0d4ded9d98ea469f8318c3e3fec0a291 in term 0.
I20260809 22:57:27.616932 12264 raft_consensus.cc:2454] T 6c2562fac19b45e1a3c542433ae1b372 P 6cda25ce8218430ea12a8a22df0c9d63 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0d4ded9d98ea469f8318c3e3fec0a291 in term 0.
I20260809 22:57:27.617712 11983 leader_election.cc:302] T 6c2562fac19b45e1a3c542433ae1b372 P 0d4ded9d98ea469f8318c3e3fec0a291 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 0d4ded9d98ea469f8318c3e3fec0a291, 6cda25ce8218430ea12a8a22df0c9d63, 819262f3c90b4e1ea104e4c962bb79ae; no voters:
I20260809 22:57:27.618131 12366 raft_consensus.cc:2781] T 6c2562fac19b45e1a3c542433ae1b372 P 0d4ded9d98ea469f8318c3e3fec0a291 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 22:57:27.618295 12366 raft_consensus.cc:482] T 6c2562fac19b45e1a3c542433ae1b372 P 0d4ded9d98ea469f8318c3e3fec0a291 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 22:57:27.618395 12366 raft_consensus.cc:3037] T 6c2562fac19b45e1a3c542433ae1b372 P 0d4ded9d98ea469f8318c3e3fec0a291 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:57:27.620925 12366 raft_consensus.cc:504] T 6c2562fac19b45e1a3c542433ae1b372 P 0d4ded9d98ea469f8318c3e3fec0a291 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d4ded9d98ea469f8318c3e3fec0a291" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 39795 } } peers { permanent_uuid: "819262f3c90b4e1ea104e4c962bb79ae" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 44053 } } peers { permanent_uuid: "6cda25ce8218430ea12a8a22df0c9d63" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 32893 } }
I20260809 22:57:27.621549 12366 leader_election.cc:288] T 6c2562fac19b45e1a3c542433ae1b372 P 0d4ded9d98ea469f8318c3e3fec0a291 [CANDIDATE]: Term 1 election: Requested vote from peers 819262f3c90b4e1ea104e4c962bb79ae (127.15.113.129:44053), 6cda25ce8218430ea12a8a22df0c9d63 (127.15.113.131:32893)
I20260809 22:57:27.621945 11879 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6c2562fac19b45e1a3c542433ae1b372" candidate_uuid: "0d4ded9d98ea469f8318c3e3fec0a291" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "819262f3c90b4e1ea104e4c962bb79ae"
I20260809 22:57:27.622048 12264 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6c2562fac19b45e1a3c542433ae1b372" candidate_uuid: "0d4ded9d98ea469f8318c3e3fec0a291" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6cda25ce8218430ea12a8a22df0c9d63"
I20260809 22:57:27.622133 11879 raft_consensus.cc:3037] T 6c2562fac19b45e1a3c542433ae1b372 P 819262f3c90b4e1ea104e4c962bb79ae [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:57:27.622272 12264 raft_consensus.cc:3037] T 6c2562fac19b45e1a3c542433ae1b372 P 6cda25ce8218430ea12a8a22df0c9d63 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:57:27.624310 11879 raft_consensus.cc:2454] T 6c2562fac19b45e1a3c542433ae1b372 P 819262f3c90b4e1ea104e4c962bb79ae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0d4ded9d98ea469f8318c3e3fec0a291 in term 1.
I20260809 22:57:27.624423 12264 raft_consensus.cc:2454] T 6c2562fac19b45e1a3c542433ae1b372 P 6cda25ce8218430ea12a8a22df0c9d63 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0d4ded9d98ea469f8318c3e3fec0a291 in term 1.
I20260809 22:57:27.624686 11986 leader_election.cc:302] T 6c2562fac19b45e1a3c542433ae1b372 P 0d4ded9d98ea469f8318c3e3fec0a291 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d4ded9d98ea469f8318c3e3fec0a291, 819262f3c90b4e1ea104e4c962bb79ae; no voters:
I20260809 22:57:27.625000 12366 raft_consensus.cc:2781] T 6c2562fac19b45e1a3c542433ae1b372 P 0d4ded9d98ea469f8318c3e3fec0a291 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 22:57:27.625439 12366 raft_consensus.cc:686] T 6c2562fac19b45e1a3c542433ae1b372 P 0d4ded9d98ea469f8318c3e3fec0a291 [term 1 LEADER]: Becoming Leader. State: Replica: 0d4ded9d98ea469f8318c3e3fec0a291, State: Running, Role: LEADER
I20260809 22:57:27.625887 12366 consensus_queue.cc:237] T 6c2562fac19b45e1a3c542433ae1b372 P 0d4ded9d98ea469f8318c3e3fec0a291 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d4ded9d98ea469f8318c3e3fec0a291" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 39795 } } peers { permanent_uuid: "819262f3c90b4e1ea104e4c962bb79ae" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 44053 } } peers { permanent_uuid: "6cda25ce8218430ea12a8a22df0c9d63" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 32893 } }
I20260809 22:57:27.630431 11659 catalog_manager.cc:5495] T 6c2562fac19b45e1a3c542433ae1b372 P 0d4ded9d98ea469f8318c3e3fec0a291 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0d4ded9d98ea469f8318c3e3fec0a291 (127.15.113.130). New cstate: current_term: 1 leader_uuid: "0d4ded9d98ea469f8318c3e3fec0a291" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d4ded9d98ea469f8318c3e3fec0a291" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 39795 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "819262f3c90b4e1ea104e4c962bb79ae" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 44053 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6cda25ce8218430ea12a8a22df0c9d63" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 32893 } health_report { overall_health: UNKNOWN } } }
I20260809 22:57:27.676829 12350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:57:27.671923 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41162 (local address 127.15.113.130:39795)
0809 22:57:27.672152 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0809 22:57:27.672157 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:57:27.672177 (+ 20us) server_negotiation.cc:407] Connection header received
0809 22:57:27.672337 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:57:27.672340 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:57:27.672418 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:57:27.672541 (+ 123us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 22:57:27.672859 (+ 318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:57:27.673616 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:57:27.674916 (+ 1300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:57:27.675290 (+ 374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:57:27.675380 (+ 90us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:57:27.675801 (+ 421us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 22:57:27.676308 (+ 507us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 22:57:27.676439 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0809 22:57:27.676510 (+ 71us) server_negotiation.cc:299] Negotiation successful
0809 22:57:27.676550 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260809 22:57:27.685376 11879 raft_consensus.cc:1261] T 6c2562fac19b45e1a3c542433ae1b372 P 819262f3c90b4e1ea104e4c962bb79ae [term 1 FOLLOWER]: Refusing update from remote peer 0d4ded9d98ea469f8318c3e3fec0a291: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 22:57:27.685601 12264 raft_consensus.cc:1261] T 6c2562fac19b45e1a3c542433ae1b372 P 6cda25ce8218430ea12a8a22df0c9d63 [term 1 FOLLOWER]: Refusing update from remote peer 0d4ded9d98ea469f8318c3e3fec0a291: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 22:57:27.686334 12370 consensus_queue.cc:1035] T 6c2562fac19b45e1a3c542433ae1b372 P 0d4ded9d98ea469f8318c3e3fec0a291 [LEADER]: Connected to new peer: Peer: permanent_uuid: "819262f3c90b4e1ea104e4c962bb79ae" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 44053 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 22:57:27.686650 12366 consensus_queue.cc:1035] T 6c2562fac19b45e1a3c542433ae1b372 P 0d4ded9d98ea469f8318c3e3fec0a291 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6cda25ce8218430ea12a8a22df0c9d63" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 32893 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 22:57:27.698395 12375 mvcc.cc:205] Tried to move back new op lower bound from 7316751350509625344 to 7316751350279409664. Current Snapshot: MvccSnapshot[applied={T|T < 7316751350509625344}]
I20260809 22:57:27.700307 12376 mvcc.cc:205] Tried to move back new op lower bound from 7316751350509625344 to 7316751350279409664. Current Snapshot: MvccSnapshot[applied={T|T < 7316751350509625344}]
I20260809 22:57:27.711243 11659 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45484:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 22:57:27.711509 11659 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45484:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 22:57:27.715464 11659 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a6315b057ded4e9cb09a53abc79dd676: Sending DeleteTablet for 3 replicas of tablet 6c2562fac19b45e1a3c542433ae1b372
I20260809 22:57:27.716620 12051 tablet_service.cc:1514] Processing DeleteTablet for tablet 6c2562fac19b45e1a3c542433ae1b372 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:57:27 UTC) from {username='kudu', principal='kudu/127.15.113.190@KRBTEST.COM'} at 127.0.0.1:41154
I20260809 22:57:27.716845 11842 tablet_service.cc:1514] Processing DeleteTablet for tablet 6c2562fac19b45e1a3c542433ae1b372 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:57:27 UTC) from {username='kudu', principal='kudu/127.15.113.190@KRBTEST.COM'} at 127.0.0.1:38964
I20260809 22:57:27.717612 12234 tablet_service.cc:1514] Processing DeleteTablet for tablet 6c2562fac19b45e1a3c542433ae1b372 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:57:27 UTC) from {username='kudu', principal='kudu/127.15.113.190@KRBTEST.COM'} at 127.0.0.1:44720
I20260809 22:57:27.721406 12384 tablet_replica.cc:330] T 6c2562fac19b45e1a3c542433ae1b372 P 0d4ded9d98ea469f8318c3e3fec0a291: stopping tablet replica
I20260809 22:57:27.722785 15814 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskItYPJR/build/asan/bin/kudu pid 11749
I20260809 22:57:27.725554 12384 raft_consensus.cc:2229] T 6c2562fac19b45e1a3c542433ae1b372 P 0d4ded9d98ea469f8318c3e3fec0a291 [term 1 LEADER]: Raft consensus shutting down.
I20260809 22:57:27.726349 12384 raft_consensus.cc:2258] T 6c2562fac19b45e1a3c542433ae1b372 P 0d4ded9d98ea469f8318c3e3fec0a291 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 22:57:27.730835 12384 ts_tablet_manager.cc:1905] T 6c2562fac19b45e1a3c542433ae1b372 P 0d4ded9d98ea469f8318c3e3fec0a291: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 22:57:27.740984 12386 tablet_replica.cc:330] T 6c2562fac19b45e1a3c542433ae1b372 P 6cda25ce8218430ea12a8a22df0c9d63: stopping tablet replica
I20260809 22:57:27.741221 12385 tablet_replica.cc:330] T 6c2562fac19b45e1a3c542433ae1b372 P 819262f3c90b4e1ea104e4c962bb79ae: stopping tablet replica
I20260809 22:57:27.741411 12386 raft_consensus.cc:2229] T 6c2562fac19b45e1a3c542433ae1b372 P 6cda25ce8218430ea12a8a22df0c9d63 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 22:57:27.741593 12385 raft_consensus.cc:2229] T 6c2562fac19b45e1a3c542433ae1b372 P 819262f3c90b4e1ea104e4c962bb79ae [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 22:57:27.741941 12386 raft_consensus.cc:2258] T 6c2562fac19b45e1a3c542433ae1b372 P 6cda25ce8218430ea12a8a22df0c9d63 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 22:57:27.742024 12385 raft_consensus.cc:2258] T 6c2562fac19b45e1a3c542433ae1b372 P 819262f3c90b4e1ea104e4c962bb79ae [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 22:57:27.744302 12385 ts_tablet_manager.cc:1905] T 6c2562fac19b45e1a3c542433ae1b372 P 819262f3c90b4e1ea104e4c962bb79ae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 22:57:27.744629 12386 ts_tablet_manager.cc:1905] T 6c2562fac19b45e1a3c542433ae1b372 P 6cda25ce8218430ea12a8a22df0c9d63: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 22:57:27.745750 12384 ts_tablet_manager.cc:1918] T 6c2562fac19b45e1a3c542433ae1b372 P 0d4ded9d98ea469f8318c3e3fec0a291: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
Aug 09 22:57:27 dist-test-slave-ncp9 krb5kdc[11600](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316245, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.113.129@KRBTEST.COM
I20260809 22:57:27.751554 12386 ts_tablet_manager.cc:1918] T 6c2562fac19b45e1a3c542433ae1b372 P 6cda25ce8218430ea12a8a22df0c9d63: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 22:57:27.751700 12386 log.cc:1196] T 6c2562fac19b45e1a3c542433ae1b372 P 6cda25ce8218430ea12a8a22df0c9d63: Deleting WAL directory at /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315935617919-15814-0/minicluster-data/ts-2/wal/wals/6c2562fac19b45e1a3c542433ae1b372
I20260809 22:57:27.752216 12386 ts_tablet_manager.cc:1939] T 6c2562fac19b45e1a3c542433ae1b372 P 6cda25ce8218430ea12a8a22df0c9d63: Deleting consensus metadata
I20260809 22:57:27.752502 12385 ts_tablet_manager.cc:1918] T 6c2562fac19b45e1a3c542433ae1b372 P 819262f3c90b4e1ea104e4c962bb79ae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 22:57:27.752681 12385 log.cc:1196] T 6c2562fac19b45e1a3c542433ae1b372 P 819262f3c90b4e1ea104e4c962bb79ae: Deleting WAL directory at /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315935617919-15814-0/minicluster-data/ts-0/wal/wals/6c2562fac19b45e1a3c542433ae1b372
I20260809 22:57:27.753021 12384 log.cc:1196] T 6c2562fac19b45e1a3c542433ae1b372 P 0d4ded9d98ea469f8318c3e3fec0a291: Deleting WAL directory at /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315935617919-15814-0/minicluster-data/ts-1/wal/wals/6c2562fac19b45e1a3c542433ae1b372
I20260809 22:57:27.753554 12384 ts_tablet_manager.cc:1939] T 6c2562fac19b45e1a3c542433ae1b372 P 0d4ded9d98ea469f8318c3e3fec0a291: Deleting consensus metadata
I20260809 22:57:27.754124 11641 catalog_manager.cc:4841] TS 6cda25ce8218430ea12a8a22df0c9d63 (127.15.113.131:32893): tablet 6c2562fac19b45e1a3c542433ae1b372 (table test-table [id=ea33194facc34a0fa181786d814542f2]) successfully deleted
I20260809 22:57:27.755025 12385 ts_tablet_manager.cc:1939] T 6c2562fac19b45e1a3c542433ae1b372 P 819262f3c90b4e1ea104e4c962bb79ae: Deleting consensus metadata
I20260809 22:57:27.755362 11640 catalog_manager.cc:4841] TS 0d4ded9d98ea469f8318c3e3fec0a291 (127.15.113.130:39795): tablet 6c2562fac19b45e1a3c542433ae1b372 (table test-table [id=ea33194facc34a0fa181786d814542f2]) successfully deleted
I20260809 22:57:27.756769 11643 catalog_manager.cc:4841] TS 819262f3c90b4e1ea104e4c962bb79ae (127.15.113.129:44053): tablet 6c2562fac19b45e1a3c542433ae1b372 (table test-table [id=ea33194facc34a0fa181786d814542f2]) successfully deleted
I20260809 22:57:27.758411 12351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:57:27.725215 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38966 (local address 127.15.113.129:44053)
0809 22:57:27.725376 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0809 22:57:27.725382 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:57:27.738192 (+ 12810us) server_negotiation.cc:407] Connection header received
0809 22:57:27.738460 (+ 268us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:57:27.738467 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:57:27.738555 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:57:27.738626 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:57:27.742182 (+ 3556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:57:27.743328 (+ 1146us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:57:27.745141 (+ 1813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:57:27.745733 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:57:27.745823 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:57:27.751373 (+ 5550us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:57:27.751388 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:57:27.751406 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:57:27.751452 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:57:27.755426 (+ 3974us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:57:27.756297 (+ 871us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:57:27.756306 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:57:27.756310 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:57:27.756489 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:57:27.756983 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:57:27.756987 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:57:27.756990 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:57:27.757312 (+ 322us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:57:27.757587 (+ 275us) server_negotiation.cc:1035] Waiting for connection context
0809 22:57:27.757896 (+ 309us) server_negotiation.cc:299] Negotiation successful
0809 22:57:27.758096 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260809 22:57:27.758960 11942 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:57:27.849398 15814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskItYPJR/build/asan/bin/kudu with pid 11749
I20260809 22:57:27.877663 15814 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskItYPJR/build/asan/bin/kudu pid 11958
Aug 09 22:57:27 dist-test-slave-ncp9 krb5kdc[11600](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316245, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.113.130@KRBTEST.COM
I20260809 22:57:27.894207 12350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:57:27.878521 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41170 (local address 127.15.113.130:39795)
0809 22:57:27.878712 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0809 22:57:27.878718 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:57:27.878745 (+ 27us) server_negotiation.cc:407] Connection header received
0809 22:57:27.878815 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:57:27.878820 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:57:27.878905 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:57:27.879041 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:57:27.880311 (+ 1270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:57:27.881595 (+ 1284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:57:27.882845 (+ 1250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:57:27.883514 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:57:27.883654 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:57:27.887701 (+ 4047us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:57:27.887715 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:57:27.887734 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:57:27.887777 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:57:27.891637 (+ 3860us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:57:27.892276 (+ 639us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:57:27.892286 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:57:27.892290 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:57:27.892452 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:57:27.892749 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:57:27.892753 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:57:27.892757 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:57:27.893192 (+ 435us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:57:27.893488 (+ 296us) server_negotiation.cc:1035] Waiting for connection context
0809 22:57:27.893661 (+ 173us) server_negotiation.cc:299] Negotiation successful
0809 22:57:27.893851 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260809 22:57:27.894965 12132 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:57:28.006100 15814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskItYPJR/build/asan/bin/kudu with pid 11958
I20260809 22:57:28.031284 15814 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskItYPJR/build/asan/bin/kudu pid 12147
Aug 09 22:57:28 dist-test-slave-ncp9 krb5kdc[11600](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316245, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.113.131@KRBTEST.COM
I20260809 22:57:28.046167 12355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:57:28.032012 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44726 (local address 127.15.113.131:32893)
0809 22:57:28.032237 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0809 22:57:28.032243 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:57:28.032264 (+ 21us) server_negotiation.cc:407] Connection header received
0809 22:57:28.032413 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:57:28.032417 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:57:28.032491 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:57:28.032574 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:57:28.033960 (+ 1386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:57:28.034673 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:57:28.036334 (+ 1661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:57:28.036719 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:57:28.036826 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:57:28.041016 (+ 4190us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:57:28.041030 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:57:28.041045 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:57:28.041077 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:57:28.043502 (+ 2425us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:57:28.044151 (+ 649us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:57:28.044160 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:57:28.044162 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:57:28.044298 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:57:28.044669 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:57:28.044673 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:57:28.044675 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:57:28.045049 (+ 374us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:57:28.045331 (+ 282us) server_negotiation.cc:1035] Waiting for connection context
0809 22:57:28.045624 (+ 293us) server_negotiation.cc:299] Negotiation successful
0809 22:57:28.045852 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94}
I20260809 22:57:28.046784 12320 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:57:28.133710 15814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskItYPJR/build/asan/bin/kudu with pid 12147
I20260809 22:57:28.156428 15814 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskItYPJR/build/asan/bin/kudu pid 11615
I20260809 22:57:28.157456 11705 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:57:28.265836 15814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskItYPJR/build/asan/bin/kudu with pid 11615
2026-08-09T22:57:28Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (4460 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-08-09T22:57:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T22:57:28Z Disabled control of system clock
I20260809 22:57:28.353788 15814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskItYPJR/build/asan/bin/kudu
/tmp/dist-test-taskItYPJR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786315935617919-15814-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786315935617919-15814-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786315935617919-15814-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786315935617919-15814-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.113.190:46739
--webserver_interface=127.15.113.190
--webserver_port=0
--builtin_ntp_servers=127.15.113.148:42135
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.113.190:46739
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 22:57:28.680835 12401 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:57:28.681125 12401 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:57:28.681241 12401 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:57:28.689260 12401 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 22:57:28.689361 12401 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:57:28.689421 12401 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:57:28.689512 12401 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 22:57:28.689567 12401 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 22:57:28.698531 12401 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.113.148:42135
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786315935617919-15814-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786315935617919-15814-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.113.190:46739
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.113.190:46739
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786315935617919-15814-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.113.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786315935617919-15814-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:57:28.699785 12401 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:57:28.701632 12401 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:57:28.713158 12408 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:57:28.713191 12410 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:57:28.713311 12414 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:57:28.713830 12401 server_base.cc:1034] running on GCE node
I20260809 22:57:28.714517 12401 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:57:28.715799 12401 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:57:28.717018 12401 hybrid_clock.cc:648] HybridClock initialized: now 1786316248716963 us; error 38 us; skew 500 ppm
I20260809 22:57:28.717360 12401 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 22:57:28.719441 12401 webserver.cc:460] Webserver started at http://127.15.113.190:40141/ using document root <none> and password file <none>
I20260809 22:57:28.720091 12401 fs_manager.cc:359] Metadata directory not provided
I20260809 22:57:28.720187 12401 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:57:28.720489 12401 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:57:28.723286 12401 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786315935617919-15814-0/minicluster-data/master-0/data/instance:
uuid: "5565421b17d949bbbd252a4e92921137"
format_stamp: "Formatted at 2026-08-09 22:57:28 on dist-test-slave-ncp9"
server_key: "454d425eb9ab8a86f9a873537d41bff2"
server_key_iv: "a909464e02561f212bc87ccf91e418aa"
server_key_version: "encryptionkey@0"
I20260809 22:57:28.724048 12401 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786315935617919-15814-0/minicluster-data/master-0/wal/instance:
uuid: "5565421b17d949bbbd252a4e92921137"
format_stamp: "Formatted at 2026-08-09 22:57:28 on dist-test-slave-ncp9"
server_key: "454d425eb9ab8a86f9a873537d41bff2"
server_key_iv: "a909464e02561f212bc87ccf91e418aa"
server_key_version: "encryptionkey@0"
I20260809 22:57:28.728794 12401 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260809 22:57:28.731953 12420 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:57:28.733178 12401 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 22:57:28.733345 12401 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786315935617919-15814-0/minicluster-data/master-0/data,/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786315935617919-15814-0/minicluster-data/master-0/wal
uuid: "5565421b17d949bbbd252a4e92921137"
format_stamp: "Formatted at 2026-08-09 22:57:28 on dist-test-slave-ncp9"
server_key: "454d425eb9ab8a86f9a873537d41bff2"
server_key_iv: "a909464e02561f212bc87ccf91e418aa"
server_key_version: "encryptionkey@0"
I20260809 22:57:28.733527 12401 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786315935617919-15814-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786315935617919-15814-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786315935617919-15814-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:57:28.777088 12401 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:57:28.777860 12401 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:57:28.778116 12401 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:57:28.794818 12401 rpc_server.cc:307] RPC server started. Bound to: 127.15.113.190:46739
I20260809 22:57:28.794850 12489 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.113.190:46739 every 8 connection(s)
I20260809 22:57:28.796033 12401 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786315935617919-15814-0/minicluster-data/master-0/data/info.pb
I20260809 22:57:28.797503 15814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskItYPJR/build/asan/bin/kudu as pid 12401
I20260809 22:57:28.797683 15814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786315935617919-15814-0/minicluster-data/master-0/wal/instance
I20260809 22:57:28.798244 15814 external_mini_cluster.cc:1428] Setting key 6f6768749381a0acd3825979576b95d8
I20260809 22:57:28.800380 12490 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:57:28.811396 12495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:57:28.800038 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49700 (local address 127.15.113.190:46739)
0809 22:57:28.800998 (+ 960us) server_negotiation.cc:206] Beginning negotiation
0809 22:57:28.801009 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 22:57:28.801050 (+ 41us) server_negotiation.cc:407] Connection header received
0809 22:57:28.802650 (+ 1600us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:57:28.802672 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:57:28.803135 (+ 463us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:57:28.803467 (+ 332us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:57:28.804052 (+ 585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:57:28.805428 (+ 1376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:57:28.807287 (+ 1859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:57:28.807905 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:57:28.808034 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:57:28.809258 (+ 1224us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:57:28.809265 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:57:28.809278 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 22:57:28.809796 (+ 518us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:57:28.809873 (+ 77us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 22:57:28.809893 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:57:28.809984 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0809 22:57:28.810214 (+ 230us) server_negotiation.cc:299] Negotiation successful
0809 22:57:28.810531 (+ 317us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":532,"thread_start_us":352,"threads_started":1}
I20260809 22:57:28.812786 12490 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5565421b17d949bbbd252a4e92921137: Bootstrap starting.
I20260809 22:57:28.815776 12490 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5565421b17d949bbbd252a4e92921137: Neither blocks nor log segments found. Creating new log.
I20260809 22:57:28.816707 12490 log.cc:824] T 00000000000000000000000000000000 P 5565421b17d949bbbd252a4e92921137: Log is configured to *not* fsync() on all Append() calls
I20260809 22:57:28.819208 12490 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5565421b17d949bbbd252a4e92921137: No bootstrap required, opened a new log
I20260809 22:57:28.826359 12490 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5565421b17d949bbbd252a4e92921137 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5565421b17d949bbbd252a4e92921137" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 46739 } }
I20260809 22:57:28.826694 12490 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5565421b17d949bbbd252a4e92921137 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:57:28.826786 12490 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5565421b17d949bbbd252a4e92921137 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5565421b17d949bbbd252a4e92921137, State: Initialized, Role: FOLLOWER
I20260809 22:57:28.827289 12490 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5565421b17d949bbbd252a4e92921137 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5565421b17d949bbbd252a4e92921137" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 46739 } }
I20260809 22:57:28.827499 12490 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5565421b17d949bbbd252a4e92921137 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 22:57:28.827631 12490 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5565421b17d949bbbd252a4e92921137 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 22:57:28.827781 12490 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5565421b17d949bbbd252a4e92921137 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:57:28.829865 12490 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5565421b17d949bbbd252a4e92921137 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5565421b17d949bbbd252a4e92921137" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 46739 } }
I20260809 22:57:28.830407 12490 leader_election.cc:302] T 00000000000000000000000000000000 P 5565421b17d949bbbd252a4e92921137 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5565421b17d949bbbd252a4e92921137; no voters:
I20260809 22:57:28.830925 12490 leader_election.cc:288] T 00000000000000000000000000000000 P 5565421b17d949bbbd252a4e92921137 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 22:57:28.831027 12497 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5565421b17d949bbbd252a4e92921137 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 22:57:28.831674 12497 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5565421b17d949bbbd252a4e92921137 [term 1 LEADER]: Becoming Leader. State: Replica: 5565421b17d949bbbd252a4e92921137, State: Running, Role: LEADER
I20260809 22:57:28.832194 12497 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5565421b17d949bbbd252a4e92921137 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5565421b17d949bbbd252a4e92921137" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 46739 } }
I20260809 22:57:28.832825 12490 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5565421b17d949bbbd252a4e92921137 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 22:57:28.834756 12502 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5565421b17d949bbbd252a4e92921137 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5565421b17d949bbbd252a4e92921137. Latest consensus state: current_term: 1 leader_uuid: "5565421b17d949bbbd252a4e92921137" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5565421b17d949bbbd252a4e92921137" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 46739 } } }
I20260809 22:57:28.834697 12499 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5565421b17d949bbbd252a4e92921137 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5565421b17d949bbbd252a4e92921137" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5565421b17d949bbbd252a4e92921137" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 46739 } } }
I20260809 22:57:28.835129 12499 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5565421b17d949bbbd252a4e92921137 [sys.catalog]: This master's current role is: LEADER
I20260809 22:57:28.835788 12502 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5565421b17d949bbbd252a4e92921137 [sys.catalog]: This master's current role is: LEADER
I20260809 22:57:28.836635 12511 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 22:57:28.841341 12511 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 22:57:28.849506 12511 catalog_manager.cc:1347] Generated new cluster ID: d46ccea9ea2f459fa153b9f851eb12fa
I20260809 22:57:28.849592 12511 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 22:57:28.864244 12511 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 22:57:28.865465 12511 catalog_manager.cc:1504] Loading token signing keys...
I20260809 22:57:28.871250 12511 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5565421b17d949bbbd252a4e92921137: Generated new TSK 0
I20260809 22:57:28.871920 12511 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 22:57:28.884016 15814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskItYPJR/build/asan/bin/kudu
/tmp/dist-test-taskItYPJR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786315935617919-15814-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786315935617919-15814-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786315935617919-15814-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786315935617919-15814-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.113.129:0
--local_ip_for_outbound_sockets=127.15.113.129
--webserver_interface=127.15.113.129
--webserver_port=0
--tserver_master_addrs=127.15.113.190:46739
--builtin_ntp_servers=127.15.113.148:42135
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 22:57:29.217752 12538 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:57:29.218016 12538 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:57:29.218138 12538 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:57:29.225221 12538 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:57:29.225328 12538 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:57:29.225478 12538 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.113.129
I20260809 22:57:29.234422 12538 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.113.148:42135
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786315935617919-15814-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786315935617919-15814-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.113.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786315935617919-15814-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.113.129
--webserver_port=0
--tserver_master_addrs=127.15.113.190:46739
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.113.129
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786315935617919-15814-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:57:29.235668 12538 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:57:29.237546 12538 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:57:29.249799 12549 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:57:29.249890 12546 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:57:29.249863 12547 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:57:29.251214 12538 server_base.cc:1034] running on GCE node
I20260809 22:57:29.251754 12538 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:57:29.252593 12538 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:57:29.253789 12538 hybrid_clock.cc:648] HybridClock initialized: now 1786316249253733 us; error 45 us; skew 500 ppm
I20260809 22:57:29.254094 12538 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 22:57:29.256167 12538 webserver.cc:460] Webserver started at http://127.15.113.129:46167/ using document root <none> and password file <none>
I20260809 22:57:29.256811 12538 fs_manager.cc:359] Metadata directory not provided
I20260809 22:57:29.256932 12538 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:57:29.257246 12538 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:57:29.259917 12538 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786315935617919-15814-0/minicluster-data/ts-0/data/instance:
uuid: "874a74185e9d40d88f543d88ba3e4d16"
format_stamp: "Formatted at 2026-08-09 22:57:29 on dist-test-slave-ncp9"
server_key: "5db54a5f9b11a48a81fa0aa58dd92d11"
server_key_iv: "4c2e8d29af1d351f631a21a6e39db932"
server_key_version: "encryptionkey@0"
I20260809 22:57:29.260681 12538 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786315935617919-15814-0/minicluster-data/ts-0/wal/instance:
uuid: "874a74185e9d40d88f543d88ba3e4d16"
format_stamp: "Formatted at 2026-08-09 22:57:29 on dist-test-slave-ncp9"
server_key: "5db54a5f9b11a48a81fa0aa58dd92d11"
server_key_iv: "4c2e8d29af1d351f631a21a6e39db932"
server_key_version: "encryptionkey@0"
I20260809 22:57:29.265594 12538 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.003s
I20260809 22:57:29.268778 12557 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:57:29.270031 12538 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 22:57:29.270179 12538 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786315935617919-15814-0/minicluster-data/ts-0/data,/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786315935617919-15814-0/minicluster-data/ts-0/wal
uuid: "874a74185e9d40d88f543d88ba3e4d16"
format_stamp: "Formatted at 2026-08-09 22:57:29 on dist-test-slave-ncp9"
server_key: "5db54a5f9b11a48a81fa0aa58dd92d11"
server_key_iv: "4c2e8d29af1d351f631a21a6e39db932"
server_key_version: "encryptionkey@0"
I20260809 22:57:29.270375 12538 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786315935617919-15814-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786315935617919-15814-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786315935617919-15814-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:57:29.287652 12538 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:57:29.288357 12538 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:57:29.288619 12538 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:57:29.289565 12538 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:57:29.290884 12538 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:57:29.290990 12538 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:57:29.291091 12538 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:57:29.291162 12538 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:57:29.321271 12538 rpc_server.cc:307] RPC server started. Bound to: 127.15.113.129:42079
I20260809 22:57:29.321334 12733 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.113.129:42079 every 8 connection(s)
I20260809 22:57:29.322515 12538 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786315935617919-15814-0/minicluster-data/ts-0/data/info.pb
I20260809 22:57:29.324319 15814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskItYPJR/build/asan/bin/kudu as pid 12538
I20260809 22:57:29.324487 15814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786315935617919-15814-0/minicluster-data/ts-0/wal/instance
I20260809 22:57:29.325068 15814 external_mini_cluster.cc:1428] Setting key 779f6075b13b8ea0abd0208fa7f3073b
I20260809 22:57:29.328161 15814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskItYPJR/build/asan/bin/kudu
/tmp/dist-test-taskItYPJR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786315935617919-15814-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786315935617919-15814-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786315935617919-15814-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786315935617919-15814-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.113.130:0
--local_ip_for_outbound_sockets=127.15.113.130
--webserver_interface=127.15.113.130
--webserver_port=0
--tserver_master_addrs=127.15.113.190:46739
--builtin_ntp_servers=127.15.113.148:42135
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260809 22:57:29.335742 12738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:57:29.324919 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.113.129:50851 (local address 127.15.113.190:46739)
0809 22:57:29.325404 (+ 485us) server_negotiation.cc:206] Beginning negotiation
0809 22:57:29.325411 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:57:29.326780 (+ 1369us) server_negotiation.cc:407] Connection header received
0809 22:57:29.329029 (+ 2249us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:57:29.329034 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:57:29.329103 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:57:29.329191 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:57:29.330466 (+ 1275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:57:29.331165 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:57:29.333141 (+ 1976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:57:29.333577 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:57:29.333695 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:57:29.335053 (+ 1358us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:57:29.335058 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:57:29.335061 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 22:57:29.335077 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:57:29.335135 (+ 58us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 22:57:29.335149 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:57:29.335217 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0809 22:57:29.335428 (+ 211us) server_negotiation.cc:299] Negotiation successful
0809 22:57:29.335502 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":360,"thread_start_us":237,"threads_started":1}
I20260809 22:57:29.337018 12744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:57:29.325531 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:46739 (local address 127.15.113.129:50851)
0809 22:57:29.326525 (+ 994us) negotiation.cc:107] Waiting for socket to connect
0809 22:57:29.326566 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0809 22:57:29.328713 (+ 2147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:57:29.329444 (+ 731us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:57:29.329455 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:57:29.329574 (+ 119us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:57:29.330322 (+ 748us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:57:29.330344 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:57:29.331314 (+ 970us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:57:29.331318 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:57:29.333018 (+ 1700us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:57:29.333033 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:57:29.333794 (+ 761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:57:29.333801 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:57:29.334060 (+ 259us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:57:29.334731 (+ 671us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 22:57:29.334734 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:57:29.334788 (+ 54us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 22:57:29.334873 (+ 85us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:57:29.335236 (+ 363us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:57:29.335247 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:57:29.335253 (+ 6us) client_negotiation.cc:769] Sending connection context
0809 22:57:29.335415 (+ 162us) client_negotiation.cc:240] Negotiation successful
0809 22:57:29.335768 (+ 353us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":556,"thread_start_us":319,"threads_started":1}
I20260809 22:57:29.338482 12735 heartbeater.cc:347] Connected to a master server at 127.15.113.190:46739
I20260809 22:57:29.338773 12735 heartbeater.cc:464] Registering TS with master...
I20260809 22:57:29.339627 12735 heartbeater.cc:511] Master 127.15.113.190:46739 requested a full tablet report, sending...
I20260809 22:57:29.341609 12444 ts_manager.cc:194] Registered new tserver with Master: 874a74185e9d40d88f543d88ba3e4d16 (127.15.113.129:42079)
I20260809 22:57:29.343483 12444 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.15.113.129:50851
W20260809 22:57:29.665970 12745 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:57:29.666244 12745 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:57:29.666338 12745 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:57:29.673322 12745 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:57:29.673419 12745 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:57:29.673542 12745 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.113.130
I20260809 22:57:29.682231 12745 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.113.148:42135
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786315935617919-15814-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786315935617919-15814-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.113.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786315935617919-15814-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.113.130
--webserver_port=0
--tserver_master_addrs=127.15.113.190:46739
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.113.130
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786315935617919-15814-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:57:29.683424 12745 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:57:29.685133 12745 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:57:29.698377 12752 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:57:29.698341 12757 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:57:29.698423 12754 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:57:29.699522 12745 server_base.cc:1034] running on GCE node
I20260809 22:57:29.700107 12745 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:57:29.701078 12745 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:57:29.702280 12745 hybrid_clock.cc:648] HybridClock initialized: now 1786316249702225 us; error 54 us; skew 500 ppm
I20260809 22:57:29.702577 12745 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 22:57:29.704741 12745 webserver.cc:460] Webserver started at http://127.15.113.130:45007/ using document root <none> and password file <none>
I20260809 22:57:29.705497 12745 fs_manager.cc:359] Metadata directory not provided
I20260809 22:57:29.705601 12745 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:57:29.705910 12745 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:57:29.708819 12745 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786315935617919-15814-0/minicluster-data/ts-1/data/instance:
uuid: "e0ae013d6387489c877ad2e47c5e2340"
format_stamp: "Formatted at 2026-08-09 22:57:29 on dist-test-slave-ncp9"
server_key: "b30e3abcfa1f52986043968082ef587e"
server_key_iv: "d5c39e7514739cf838516c3df2058b0f"
server_key_version: "encryptionkey@0"
I20260809 22:57:29.709692 12745 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786315935617919-15814-0/minicluster-data/ts-1/wal/instance:
uuid: "e0ae013d6387489c877ad2e47c5e2340"
format_stamp: "Formatted at 2026-08-09 22:57:29 on dist-test-slave-ncp9"
server_key: "b30e3abcfa1f52986043968082ef587e"
server_key_iv: "d5c39e7514739cf838516c3df2058b0f"
server_key_version: "encryptionkey@0"
I20260809 22:57:29.714677 12745 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 22:57:29.718017 12766 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:57:29.719283 12745 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260809 22:57:29.719437 12745 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786315935617919-15814-0/minicluster-data/ts-1/data,/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786315935617919-15814-0/minicluster-data/ts-1/wal
uuid: "e0ae013d6387489c877ad2e47c5e2340"
format_stamp: "Formatted at 2026-08-09 22:57:29 on dist-test-slave-ncp9"
server_key: "b30e3abcfa1f52986043968082ef587e"
server_key_iv: "d5c39e7514739cf838516c3df2058b0f"
server_key_version: "encryptionkey@0"
I20260809 22:57:29.719636 12745 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786315935617919-15814-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786315935617919-15814-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786315935617919-15814-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:57:29.736688 12745 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:57:29.737528 12745 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:57:29.737805 12745 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:57:29.738751 12745 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:57:29.740118 12745 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:57:29.740226 12745 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:57:29.740334 12745 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:57:29.740403 12745 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:57:29.769487 12745 rpc_server.cc:307] RPC server started. Bound to: 127.15.113.130:37919
I20260809 22:57:29.769536 12933 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.113.130:37919 every 8 connection(s)
I20260809 22:57:29.770743 12745 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786315935617919-15814-0/minicluster-data/ts-1/data/info.pb
I20260809 22:57:29.779382 15814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskItYPJR/build/asan/bin/kudu as pid 12745
I20260809 22:57:29.779542 15814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786315935617919-15814-0/minicluster-data/ts-1/wal/instance
I20260809 22:57:29.779983 15814 external_mini_cluster.cc:1428] Setting key 99241096d03578b24a69bcaaa8c57254
I20260809 22:57:29.782819 12738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:57:29.773099 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.113.130:54557 (local address 127.15.113.190:46739)
0809 22:57:29.773322 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0809 22:57:29.773329 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:57:29.774803 (+ 1474us) server_negotiation.cc:407] Connection header received
0809 22:57:29.776478 (+ 1675us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:57:29.776482 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:57:29.776546 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:57:29.776630 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:57:29.777662 (+ 1032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:57:29.778356 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:57:29.779937 (+ 1581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:57:29.780557 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:57:29.780678 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:57:29.781998 (+ 1320us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:57:29.782004 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:57:29.782009 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 22:57:29.782030 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:57:29.782072 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 22:57:29.782088 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:57:29.782177 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0809 22:57:29.782409 (+ 232us) server_negotiation.cc:299] Negotiation successful
0809 22:57:29.782505 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260809 22:57:29.782914 15814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskItYPJR/build/asan/bin/kudu
/tmp/dist-test-taskItYPJR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786315935617919-15814-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786315935617919-15814-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786315935617919-15814-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786315935617919-15814-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.113.131:0
--local_ip_for_outbound_sockets=127.15.113.131
--webserver_interface=127.15.113.131
--webserver_port=0
--tserver_master_addrs=127.15.113.190:46739
--builtin_ntp_servers=127.15.113.148:42135
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260809 22:57:29.783672 12938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:57:29.773705 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:46739 (local address 127.15.113.130:54557)
0809 22:57:29.774640 (+ 935us) negotiation.cc:107] Waiting for socket to connect
0809 22:57:29.774683 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 22:57:29.776254 (+ 1571us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:57:29.776826 (+ 572us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:57:29.776834 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:57:29.776956 (+ 122us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:57:29.777529 (+ 573us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:57:29.777548 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:57:29.778462 (+ 914us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:57:29.778466 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:57:29.779793 (+ 1327us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:57:29.779806 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:57:29.780794 (+ 988us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:57:29.780802 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:57:29.781044 (+ 242us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:57:29.781696 (+ 652us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 22:57:29.781698 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:57:29.781736 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 22:57:29.781809 (+ 73us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:57:29.782209 (+ 400us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:57:29.782219 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:57:29.782225 (+ 6us) client_negotiation.cc:769] Sending connection context
0809 22:57:29.782358 (+ 133us) client_negotiation.cc:240] Negotiation successful
0809 22:57:29.782641 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":437,"thread_start_us":250,"threads_started":1}
I20260809 22:57:29.784940 12934 heartbeater.cc:347] Connected to a master server at 127.15.113.190:46739
I20260809 22:57:29.785246 12934 heartbeater.cc:464] Registering TS with master...
I20260809 22:57:29.786069 12934 heartbeater.cc:511] Master 127.15.113.190:46739 requested a full tablet report, sending...
I20260809 22:57:29.787473 12444 ts_manager.cc:194] Registered new tserver with Master: e0ae013d6387489c877ad2e47c5e2340 (127.15.113.130:37919)
I20260809 22:57:29.788259 12444 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.15.113.130:54557
W20260809 22:57:30.118916 12939 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:57:30.119206 12939 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:57:30.119349 12939 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:57:30.126585 12939 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:57:30.126693 12939 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:57:30.126847 12939 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.113.131
I20260809 22:57:30.135819 12939 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.113.148:42135
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786315935617919-15814-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786315935617919-15814-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.113.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786315935617919-15814-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.113.131
--webserver_port=0
--tserver_master_addrs=127.15.113.190:46739
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.113.131
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786315935617919-15814-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:57:30.137140 12939 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:57:30.138942 12939 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:57:30.151678 12949 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:57:30.151714 12950 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:57:30.151630 12953 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:57:30.152623 12939 server_base.cc:1034] running on GCE node
I20260809 22:57:30.153210 12939 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:57:30.154070 12939 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:57:30.155264 12939 hybrid_clock.cc:648] HybridClock initialized: now 1786316250155223 us; error 39 us; skew 500 ppm
I20260809 22:57:30.155542 12939 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 22:57:30.157776 12939 webserver.cc:460] Webserver started at http://127.15.113.131:38909/ using document root <none> and password file <none>
I20260809 22:57:30.158420 12939 fs_manager.cc:359] Metadata directory not provided
I20260809 22:57:30.158520 12939 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:57:30.158803 12939 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:57:30.161710 12939 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786315935617919-15814-0/minicluster-data/ts-2/data/instance:
uuid: "2b636635d7764b9a84546f7844221dde"
format_stamp: "Formatted at 2026-08-09 22:57:30 on dist-test-slave-ncp9"
server_key: "ce887b8fc0774058ce3407469aa2d322"
server_key_iv: "e0f4b227f70dc9fccaeca6ab9f140de6"
server_key_version: "encryptionkey@0"
I20260809 22:57:30.162518 12939 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786315935617919-15814-0/minicluster-data/ts-2/wal/instance:
uuid: "2b636635d7764b9a84546f7844221dde"
format_stamp: "Formatted at 2026-08-09 22:57:30 on dist-test-slave-ncp9"
server_key: "ce887b8fc0774058ce3407469aa2d322"
server_key_iv: "e0f4b227f70dc9fccaeca6ab9f140de6"
server_key_version: "encryptionkey@0"
I20260809 22:57:30.167665 12939 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260809 22:57:30.171053 12965 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:57:30.172298 12939 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 22:57:30.172451 12939 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786315935617919-15814-0/minicluster-data/ts-2/data,/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786315935617919-15814-0/minicluster-data/ts-2/wal
uuid: "2b636635d7764b9a84546f7844221dde"
format_stamp: "Formatted at 2026-08-09 22:57:30 on dist-test-slave-ncp9"
server_key: "ce887b8fc0774058ce3407469aa2d322"
server_key_iv: "e0f4b227f70dc9fccaeca6ab9f140de6"
server_key_version: "encryptionkey@0"
I20260809 22:57:30.172663 12939 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786315935617919-15814-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786315935617919-15814-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786315935617919-15814-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:57:30.210436 12939 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:57:30.211256 12939 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:57:30.211570 12939 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:57:30.212538 12939 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:57:30.214049 12939 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:57:30.214150 12939 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:57:30.214263 12939 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:57:30.214337 12939 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:57:30.245980 12939 rpc_server.cc:307] RPC server started. Bound to: 127.15.113.131:45137
I20260809 22:57:30.246021 13140 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.113.131:45137 every 8 connection(s)
I20260809 22:57:30.247361 12939 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786315935617919-15814-0/minicluster-data/ts-2/data/info.pb
I20260809 22:57:30.247812 15814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskItYPJR/build/asan/bin/kudu as pid 12939
I20260809 22:57:30.247969 15814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786315935617919-15814-0/minicluster-data/ts-2/wal/instance
I20260809 22:57:30.248524 15814 external_mini_cluster.cc:1428] Setting key e4a251a5ea5d6a72e41e2d6cb088f908
I20260809 22:57:30.259178 12738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:57:30.249996 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.113.131:48391 (local address 127.15.113.190:46739)
0809 22:57:30.250209 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0809 22:57:30.250217 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 22:57:30.251820 (+ 1603us) server_negotiation.cc:407] Connection header received
0809 22:57:30.253569 (+ 1749us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:57:30.253573 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:57:30.253643 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:57:30.253736 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:57:30.254722 (+ 986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:57:30.255457 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:57:30.256932 (+ 1475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:57:30.257343 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:57:30.257441 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:57:30.258567 (+ 1126us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:57:30.258572 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:57:30.258575 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 22:57:30.258592 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:57:30.258623 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 22:57:30.258634 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:57:30.258700 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0809 22:57:30.258856 (+ 156us) server_negotiation.cc:299] Negotiation successful
0809 22:57:30.258929 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260809 22:57:30.260016 13144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:57:30.250663 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:46739 (local address 127.15.113.131:48391)
0809 22:57:30.251673 (+ 1010us) negotiation.cc:107] Waiting for socket to connect
0809 22:57:30.251717 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 22:57:30.253337 (+ 1620us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:57:30.253902 (+ 565us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:57:30.253910 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:57:30.253990 (+ 80us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:57:30.254531 (+ 541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:57:30.254548 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:57:30.255590 (+ 1042us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:57:30.255593 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:57:30.256795 (+ 1202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:57:30.256807 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:57:30.257518 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:57:30.257522 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:57:30.257733 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:57:30.258299 (+ 566us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 22:57:30.258301 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:57:30.258337 (+ 36us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 22:57:30.258410 (+ 73us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:57:30.258704 (+ 294us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:57:30.258710 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:57:30.258714 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 22:57:30.258841 (+ 127us) client_negotiation.cc:240] Negotiation successful
0809 22:57:30.259097 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":492,"thread_start_us":310,"threads_started":1}
I20260809 22:57:30.261219 13141 heartbeater.cc:347] Connected to a master server at 127.15.113.190:46739
I20260809 22:57:30.261469 13141 heartbeater.cc:464] Registering TS with master...
I20260809 22:57:30.262141 13141 heartbeater.cc:511] Master 127.15.113.190:46739 requested a full tablet report, sending...
I20260809 22:57:30.263523 12443 ts_manager.cc:194] Registered new tserver with Master: 2b636635d7764b9a84546f7844221dde (127.15.113.131:45137)
I20260809 22:57:30.264317 12443 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.15.113.131:48391
I20260809 22:57:30.275226 15814 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 22:57:30.284399 12738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:57:30.279079 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49710 (local address 127.15.113.190:46739)
0809 22:57:30.279310 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0809 22:57:30.279317 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:57:30.279687 (+ 370us) server_negotiation.cc:407] Connection header received
0809 22:57:30.279802 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:57:30.279805 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:57:30.279869 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:57:30.279941 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:57:30.280352 (+ 411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:57:30.281033 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:57:30.282331 (+ 1298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:57:30.282812 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:57:30.282901 (+ 89us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:57:30.283813 (+ 912us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:57:30.283817 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:57:30.283821 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 22:57:30.283835 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:57:30.283860 (+ 25us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 22:57:30.283870 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:57:30.283954 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0809 22:57:30.284081 (+ 127us) server_negotiation.cc:299] Negotiation successful
0809 22:57:30.284123 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260809 22:57:30.287739 12443 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49710:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 22:57:30.294198 12443 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 22:57:30.315119 13161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:57:30.306210 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53390 (local address 127.15.113.130:37919)
0809 22:57:30.307231 (+ 1021us) server_negotiation.cc:206] Beginning negotiation
0809 22:57:30.307241 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 22:57:30.307269 (+ 28us) server_negotiation.cc:407] Connection header received
0809 22:57:30.307368 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:57:30.307376 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:57:30.307645 (+ 269us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:57:30.308460 (+ 815us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:57:30.308697 (+ 237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:57:30.310425 (+ 1728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:57:30.312754 (+ 2329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:57:30.314019 (+ 1265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:57:30.314162 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:57:30.314381 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0809 22:57:30.314633 (+ 252us) server_negotiation.cc:299] Negotiation successful
0809 22:57:30.314791 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":860,"thread_start_us":746,"threads_started":1}
I20260809 22:57:30.315106 13159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:57:30.305649 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.130:37919 (local address 127.0.0.1:53390)
0809 22:57:30.306607 (+ 958us) negotiation.cc:107] Waiting for socket to connect
0809 22:57:30.306623 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 22:57:30.306728 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:57:30.307887 (+ 1159us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:57:30.307894 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:57:30.307926 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:57:30.308272 (+ 346us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:57:30.308284 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:57:30.310582 (+ 2298us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:57:30.310588 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:57:30.312499 (+ 1911us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:57:30.312512 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:57:30.314203 (+ 1691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:57:30.314210 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:57:30.314411 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:57:30.314460 (+ 49us) client_negotiation.cc:769] Sending connection context
0809 22:57:30.314566 (+ 106us) client_negotiation.cc:240] Negotiation successful
0809 22:57:30.314730 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":847,"mutex_wait_us":88,"thread_start_us":703,"threads_started":1}
I20260809 22:57:30.315798 13157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:57:30.304914 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.131:45137 (local address 127.0.0.1:47620)
0809 22:57:30.305835 (+ 921us) negotiation.cc:107] Waiting for socket to connect
0809 22:57:30.305880 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0809 22:57:30.306039 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:57:30.308412 (+ 2373us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:57:30.308417 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:57:30.308441 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:57:30.308997 (+ 556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:57:30.309010 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:57:30.310773 (+ 1763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:57:30.310778 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:57:30.313240 (+ 2462us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:57:30.313256 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:57:30.315083 (+ 1827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:57:30.315089 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:57:30.315243 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:57:30.315274 (+ 31us) client_negotiation.cc:769] Sending connection context
0809 22:57:30.315342 (+ 68us) client_negotiation.cc:240] Negotiation successful
0809 22:57:30.315457 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":662,"mutex_wait_us":63,"thread_start_us":524,"threads_started":1}
I20260809 22:57:30.316339 13162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:57:30.306735 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42368 (local address 127.15.113.129:42079)
0809 22:57:30.308068 (+ 1333us) server_negotiation.cc:206] Beginning negotiation
0809 22:57:30.308074 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:57:30.308096 (+ 22us) server_negotiation.cc:407] Connection header received
0809 22:57:30.308171 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:57:30.308176 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:57:30.308333 (+ 157us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:57:30.308710 (+ 377us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:57:30.309448 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:57:30.310870 (+ 1422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:57:30.313490 (+ 2620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:57:30.314857 (+ 1367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:57:30.315007 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:57:30.315221 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0809 22:57:30.315893 (+ 672us) server_negotiation.cc:299] Negotiation successful
0809 22:57:30.316059 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1203,"thread_start_us":1091,"threads_started":1}
I20260809 22:57:30.316417 13158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:57:30.305115 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47620 (local address 127.15.113.131:45137)
0809 22:57:30.307842 (+ 2727us) server_negotiation.cc:206] Beginning negotiation
0809 22:57:30.307849 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:57:30.307882 (+ 33us) server_negotiation.cc:407] Connection header received
0809 22:57:30.307965 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:57:30.307972 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:57:30.308212 (+ 240us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:57:30.308408 (+ 196us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:57:30.309237 (+ 829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:57:30.310649 (+ 1412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:57:30.313484 (+ 2835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:57:30.314857 (+ 1373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:57:30.315626 (+ 769us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:57:30.315825 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0809 22:57:30.315906 (+ 81us) server_negotiation.cc:299] Negotiation successful
0809 22:57:30.316060 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2560,"thread_start_us":2402,"threads_started":1}
I20260809 22:57:30.316694 13160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:57:30.306228 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.129:42079 (local address 127.0.0.1:42368)
0809 22:57:30.307193 (+ 965us) negotiation.cc:107] Waiting for socket to connect
0809 22:57:30.307215 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0809 22:57:30.307319 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:57:30.308711 (+ 1392us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:57:30.308716 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:57:30.308741 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:57:30.309227 (+ 486us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:57:30.309242 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:57:30.311013 (+ 1771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:57:30.311021 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:57:30.313267 (+ 2246us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:57:30.313277 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:57:30.315349 (+ 2072us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:57:30.315355 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:57:30.315507 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:57:30.315537 (+ 30us) client_negotiation.cc:769] Sending connection context
0809 22:57:30.315609 (+ 72us) client_negotiation.cc:240] Negotiation successful
0809 22:57:30.315717 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":853,"mutex_wait_us":17,"thread_start_us":705,"threads_started":1}
I20260809 22:57:30.319022 12840 tablet_service.cc:1467] Processing CreateTablet for tablet 1202abbb4862475c8a1085ff59b0821d (DEFAULT_TABLE table=test-table [id=ab1fef4477bd4569bce8ed5657cc81d4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:57:30.319706 12630 tablet_service.cc:1467] Processing CreateTablet for tablet 1202abbb4862475c8a1085ff59b0821d (DEFAULT_TABLE table=test-table [id=ab1fef4477bd4569bce8ed5657cc81d4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:57:30.320364 12840 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1202abbb4862475c8a1085ff59b0821d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:57:30.320981 12630 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1202abbb4862475c8a1085ff59b0821d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:57:30.323033 13038 tablet_service.cc:1467] Processing CreateTablet for tablet 1202abbb4862475c8a1085ff59b0821d (DEFAULT_TABLE table=test-table [id=ab1fef4477bd4569bce8ed5657cc81d4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:57:30.324430 13038 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1202abbb4862475c8a1085ff59b0821d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:57:30.334568 13163 tablet_bootstrap.cc:492] T 1202abbb4862475c8a1085ff59b0821d P e0ae013d6387489c877ad2e47c5e2340: Bootstrap starting.
I20260809 22:57:30.336333 13164 tablet_bootstrap.cc:492] T 1202abbb4862475c8a1085ff59b0821d P 874a74185e9d40d88f543d88ba3e4d16: Bootstrap starting.
I20260809 22:57:30.338629 13163 tablet_bootstrap.cc:654] T 1202abbb4862475c8a1085ff59b0821d P e0ae013d6387489c877ad2e47c5e2340: Neither blocks nor log segments found. Creating new log.
I20260809 22:57:30.339875 13164 tablet_bootstrap.cc:654] T 1202abbb4862475c8a1085ff59b0821d P 874a74185e9d40d88f543d88ba3e4d16: Neither blocks nor log segments found. Creating new log.
I20260809 22:57:30.339919 13163 log.cc:824] T 1202abbb4862475c8a1085ff59b0821d P e0ae013d6387489c877ad2e47c5e2340: Log is configured to *not* fsync() on all Append() calls
I20260809 22:57:30.340339 13167 tablet_bootstrap.cc:492] T 1202abbb4862475c8a1085ff59b0821d P 2b636635d7764b9a84546f7844221dde: Bootstrap starting.
I20260809 22:57:30.341146 13164 log.cc:824] T 1202abbb4862475c8a1085ff59b0821d P 874a74185e9d40d88f543d88ba3e4d16: Log is configured to *not* fsync() on all Append() calls
I20260809 22:57:30.343281 13167 tablet_bootstrap.cc:654] T 1202abbb4862475c8a1085ff59b0821d P 2b636635d7764b9a84546f7844221dde: Neither blocks nor log segments found. Creating new log.
I20260809 22:57:30.344173 13164 tablet_bootstrap.cc:492] T 1202abbb4862475c8a1085ff59b0821d P 874a74185e9d40d88f543d88ba3e4d16: No bootstrap required, opened a new log
I20260809 22:57:30.344297 13167 log.cc:824] T 1202abbb4862475c8a1085ff59b0821d P 2b636635d7764b9a84546f7844221dde: Log is configured to *not* fsync() on all Append() calls
I20260809 22:57:30.344424 13164 ts_tablet_manager.cc:1397] T 1202abbb4862475c8a1085ff59b0821d P 874a74185e9d40d88f543d88ba3e4d16: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260809 22:57:30.346033 13163 tablet_bootstrap.cc:492] T 1202abbb4862475c8a1085ff59b0821d P e0ae013d6387489c877ad2e47c5e2340: No bootstrap required, opened a new log
I20260809 22:57:30.346347 12735 heartbeater.cc:503] Master 127.15.113.190:46739 was elected leader, sending a full tablet report...
I20260809 22:57:30.346992 13167 tablet_bootstrap.cc:492] T 1202abbb4862475c8a1085ff59b0821d P 2b636635d7764b9a84546f7844221dde: No bootstrap required, opened a new log
I20260809 22:57:30.347285 13167 ts_tablet_manager.cc:1397] T 1202abbb4862475c8a1085ff59b0821d P 2b636635d7764b9a84546f7844221dde: Time spent bootstrapping tablet: real 0.007s user 0.007s sys 0.000s
I20260809 22:57:30.348068 13163 ts_tablet_manager.cc:1397] T 1202abbb4862475c8a1085ff59b0821d P e0ae013d6387489c877ad2e47c5e2340: Time spent bootstrapping tablet: real 0.014s user 0.005s sys 0.005s
I20260809 22:57:30.354432 13164 raft_consensus.cc:348] T 1202abbb4862475c8a1085ff59b0821d P 874a74185e9d40d88f543d88ba3e4d16 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b636635d7764b9a84546f7844221dde" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 45137 } } peers { permanent_uuid: "874a74185e9d40d88f543d88ba3e4d16" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 42079 } } peers { permanent_uuid: "e0ae013d6387489c877ad2e47c5e2340" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 37919 } }
I20260809 22:57:30.354847 13164 raft_consensus.cc:374] T 1202abbb4862475c8a1085ff59b0821d P 874a74185e9d40d88f543d88ba3e4d16 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:57:30.354712 13167 raft_consensus.cc:348] T 1202abbb4862475c8a1085ff59b0821d P 2b636635d7764b9a84546f7844221dde [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b636635d7764b9a84546f7844221dde" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 45137 } } peers { permanent_uuid: "874a74185e9d40d88f543d88ba3e4d16" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 42079 } } peers { permanent_uuid: "e0ae013d6387489c877ad2e47c5e2340" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 37919 } }
I20260809 22:57:30.354979 13164 raft_consensus.cc:729] T 1202abbb4862475c8a1085ff59b0821d P 874a74185e9d40d88f543d88ba3e4d16 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 874a74185e9d40d88f543d88ba3e4d16, State: Initialized, Role: FOLLOWER
I20260809 22:57:30.355098 13167 raft_consensus.cc:374] T 1202abbb4862475c8a1085ff59b0821d P 2b636635d7764b9a84546f7844221dde [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:57:30.355207 13167 raft_consensus.cc:729] T 1202abbb4862475c8a1085ff59b0821d P 2b636635d7764b9a84546f7844221dde [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b636635d7764b9a84546f7844221dde, State: Initialized, Role: FOLLOWER
I20260809 22:57:30.355640 13164 consensus_queue.cc:260] T 1202abbb4862475c8a1085ff59b0821d P 874a74185e9d40d88f543d88ba3e4d16 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b636635d7764b9a84546f7844221dde" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 45137 } } peers { permanent_uuid: "874a74185e9d40d88f543d88ba3e4d16" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 42079 } } peers { permanent_uuid: "e0ae013d6387489c877ad2e47c5e2340" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 37919 } }
I20260809 22:57:30.355758 13167 consensus_queue.cc:260] T 1202abbb4862475c8a1085ff59b0821d P 2b636635d7764b9a84546f7844221dde [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b636635d7764b9a84546f7844221dde" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 45137 } } peers { permanent_uuid: "874a74185e9d40d88f543d88ba3e4d16" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 42079 } } peers { permanent_uuid: "e0ae013d6387489c877ad2e47c5e2340" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 37919 } }
I20260809 22:57:30.357326 13164 ts_tablet_manager.cc:1428] T 1202abbb4862475c8a1085ff59b0821d P 874a74185e9d40d88f543d88ba3e4d16: Time spent starting tablet: real 0.013s user 0.010s sys 0.005s
I20260809 22:57:30.358223 13141 heartbeater.cc:503] Master 127.15.113.190:46739 was elected leader, sending a full tablet report...
I20260809 22:57:30.358423 13163 raft_consensus.cc:348] T 1202abbb4862475c8a1085ff59b0821d P e0ae013d6387489c877ad2e47c5e2340 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b636635d7764b9a84546f7844221dde" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 45137 } } peers { permanent_uuid: "874a74185e9d40d88f543d88ba3e4d16" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 42079 } } peers { permanent_uuid: "e0ae013d6387489c877ad2e47c5e2340" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 37919 } }
I20260809 22:57:30.358804 13163 raft_consensus.cc:374] T 1202abbb4862475c8a1085ff59b0821d P e0ae013d6387489c877ad2e47c5e2340 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:57:30.358860 13167 ts_tablet_manager.cc:1428] T 1202abbb4862475c8a1085ff59b0821d P 2b636635d7764b9a84546f7844221dde: Time spent starting tablet: real 0.011s user 0.007s sys 0.004s
I20260809 22:57:30.358891 13163 raft_consensus.cc:729] T 1202abbb4862475c8a1085ff59b0821d P e0ae013d6387489c877ad2e47c5e2340 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e0ae013d6387489c877ad2e47c5e2340, State: Initialized, Role: FOLLOWER
I20260809 22:57:30.359457 13163 consensus_queue.cc:260] T 1202abbb4862475c8a1085ff59b0821d P e0ae013d6387489c877ad2e47c5e2340 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b636635d7764b9a84546f7844221dde" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 45137 } } peers { permanent_uuid: "874a74185e9d40d88f543d88ba3e4d16" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 42079 } } peers { permanent_uuid: "e0ae013d6387489c877ad2e47c5e2340" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 37919 } }
I20260809 22:57:30.360522 12934 heartbeater.cc:503] Master 127.15.113.190:46739 was elected leader, sending a full tablet report...
I20260809 22:57:30.362303 13163 ts_tablet_manager.cc:1428] T 1202abbb4862475c8a1085ff59b0821d P e0ae013d6387489c877ad2e47c5e2340: Time spent starting tablet: real 0.014s user 0.012s sys 0.000s
W20260809 22:57:30.499085 13142 tablet.cc:2369] T 1202abbb4862475c8a1085ff59b0821d P 2b636635d7764b9a84546f7844221dde: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 22:57:30.522641 12937 tablet.cc:2369] T 1202abbb4862475c8a1085ff59b0821d P e0ae013d6387489c877ad2e47c5e2340: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 22:57:30.553220 13173 raft_consensus.cc:482] T 1202abbb4862475c8a1085ff59b0821d P 874a74185e9d40d88f543d88ba3e4d16 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 22:57:30.553517 13173 raft_consensus.cc:504] T 1202abbb4862475c8a1085ff59b0821d P 874a74185e9d40d88f543d88ba3e4d16 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b636635d7764b9a84546f7844221dde" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 45137 } } peers { permanent_uuid: "874a74185e9d40d88f543d88ba3e4d16" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 42079 } } peers { permanent_uuid: "e0ae013d6387489c877ad2e47c5e2340" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 37919 } }
I20260809 22:57:30.555207 13173 leader_election.cc:288] T 1202abbb4862475c8a1085ff59b0821d P 874a74185e9d40d88f543d88ba3e4d16 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2b636635d7764b9a84546f7844221dde (127.15.113.131:45137), e0ae013d6387489c877ad2e47c5e2340 (127.15.113.130:37919)
I20260809 22:57:30.561909 13158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:57:30.555564 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.113.129:54677 (local address 127.15.113.131:45137)
0809 22:57:30.555748 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0809 22:57:30.555754 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:57:30.556616 (+ 862us) server_negotiation.cc:407] Connection header received
0809 22:57:30.556776 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:57:30.556783 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:57:30.556865 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:57:30.556969 (+ 104us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:57:30.557571 (+ 602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:57:30.558330 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:57:30.560257 (+ 1927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:57:30.561064 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:57:30.561232 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:57:30.561299 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0809 22:57:30.561562 (+ 263us) server_negotiation.cc:299] Negotiation successful
0809 22:57:30.561678 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260809 22:57:30.561918 13161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:57:30.555899 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.113.129:46389 (local address 127.15.113.130:37919)
0809 22:57:30.556090 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0809 22:57:30.556096 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:57:30.556760 (+ 664us) server_negotiation.cc:407] Connection header received
0809 22:57:30.556916 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:57:30.556920 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:57:30.557011 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:57:30.557108 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:57:30.557686 (+ 578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:57:30.558426 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:57:30.560274 (+ 1848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:57:30.561030 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:57:30.561144 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:57:30.561211 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0809 22:57:30.561535 (+ 324us) server_negotiation.cc:299] Negotiation successful
0809 22:57:30.561668 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260809 22:57:30.562055 13178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:57:30.555712 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.130:37919 (local address 127.15.113.129:46389)
0809 22:57:30.556633 (+ 921us) negotiation.cc:107] Waiting for socket to connect
0809 22:57:30.556650 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 22:57:30.556784 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:57:30.557139 (+ 355us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:57:30.557143 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:57:30.557173 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:57:30.557524 (+ 351us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:57:30.557535 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:57:30.558560 (+ 1025us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:57:30.558564 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:57:30.560063 (+ 1499us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:57:30.560076 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:57:30.561204 (+ 1128us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:57:30.561210 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:57:30.561369 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:57:30.561408 (+ 39us) client_negotiation.cc:769] Sending connection context
0809 22:57:30.561481 (+ 73us) client_negotiation.cc:240] Negotiation successful
0809 22:57:30.561596 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":796,"mutex_wait_us":7,"thread_start_us":500,"threads_started":1}
I20260809 22:57:30.562108 13177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:57:30.555656 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.131:45137 (local address 127.15.113.129:54677)
0809 22:57:30.556497 (+ 841us) negotiation.cc:107] Waiting for socket to connect
0809 22:57:30.556516 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 22:57:30.556638 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:57:30.557001 (+ 363us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:57:30.557005 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:57:30.557029 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:57:30.557421 (+ 392us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:57:30.557434 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:57:30.558453 (+ 1019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:57:30.558458 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:57:30.560057 (+ 1599us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:57:30.560070 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:57:30.561269 (+ 1199us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:57:30.561274 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:57:30.561381 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:57:30.561428 (+ 47us) client_negotiation.cc:769] Sending connection context
0809 22:57:30.561514 (+ 86us) client_negotiation.cc:240] Negotiation successful
0809 22:57:30.561641 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":723,"mutex_wait_us":95,"thread_start_us":459,"threads_started":1}
I20260809 22:57:30.563169 12873 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1202abbb4862475c8a1085ff59b0821d" candidate_uuid: "874a74185e9d40d88f543d88ba3e4d16" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e0ae013d6387489c877ad2e47c5e2340" is_pre_election: true
I20260809 22:57:30.563197 13073 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1202abbb4862475c8a1085ff59b0821d" candidate_uuid: "874a74185e9d40d88f543d88ba3e4d16" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b636635d7764b9a84546f7844221dde" is_pre_election: true
I20260809 22:57:30.563669 13073 raft_consensus.cc:2454] T 1202abbb4862475c8a1085ff59b0821d P 2b636635d7764b9a84546f7844221dde [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 874a74185e9d40d88f543d88ba3e4d16 in term 0.
I20260809 22:57:30.563664 12873 raft_consensus.cc:2454] T 1202abbb4862475c8a1085ff59b0821d P e0ae013d6387489c877ad2e47c5e2340 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 874a74185e9d40d88f543d88ba3e4d16 in term 0.
I20260809 22:57:30.564260 12560 leader_election.cc:302] T 1202abbb4862475c8a1085ff59b0821d P 874a74185e9d40d88f543d88ba3e4d16 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2b636635d7764b9a84546f7844221dde, 874a74185e9d40d88f543d88ba3e4d16; no voters:
I20260809 22:57:30.564617 13173 raft_consensus.cc:2781] T 1202abbb4862475c8a1085ff59b0821d P 874a74185e9d40d88f543d88ba3e4d16 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 22:57:30.564785 13173 raft_consensus.cc:482] T 1202abbb4862475c8a1085ff59b0821d P 874a74185e9d40d88f543d88ba3e4d16 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 22:57:30.564913 13173 raft_consensus.cc:3037] T 1202abbb4862475c8a1085ff59b0821d P 874a74185e9d40d88f543d88ba3e4d16 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:57:30.567023 13173 raft_consensus.cc:504] T 1202abbb4862475c8a1085ff59b0821d P 874a74185e9d40d88f543d88ba3e4d16 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b636635d7764b9a84546f7844221dde" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 45137 } } peers { permanent_uuid: "874a74185e9d40d88f543d88ba3e4d16" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 42079 } } peers { permanent_uuid: "e0ae013d6387489c877ad2e47c5e2340" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 37919 } }
I20260809 22:57:30.567597 13173 leader_election.cc:288] T 1202abbb4862475c8a1085ff59b0821d P 874a74185e9d40d88f543d88ba3e4d16 [CANDIDATE]: Term 1 election: Requested vote from peers 2b636635d7764b9a84546f7844221dde (127.15.113.131:45137), e0ae013d6387489c877ad2e47c5e2340 (127.15.113.130:37919)
I20260809 22:57:30.568011 13073 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1202abbb4862475c8a1085ff59b0821d" candidate_uuid: "874a74185e9d40d88f543d88ba3e4d16" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b636635d7764b9a84546f7844221dde"
I20260809 22:57:30.568025 12873 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1202abbb4862475c8a1085ff59b0821d" candidate_uuid: "874a74185e9d40d88f543d88ba3e4d16" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e0ae013d6387489c877ad2e47c5e2340"
I20260809 22:57:30.568193 13073 raft_consensus.cc:3037] T 1202abbb4862475c8a1085ff59b0821d P 2b636635d7764b9a84546f7844221dde [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:57:30.568192 12873 raft_consensus.cc:3037] T 1202abbb4862475c8a1085ff59b0821d P e0ae013d6387489c877ad2e47c5e2340 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:57:30.570327 13073 raft_consensus.cc:2454] T 1202abbb4862475c8a1085ff59b0821d P 2b636635d7764b9a84546f7844221dde [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 874a74185e9d40d88f543d88ba3e4d16 in term 1.
I20260809 22:57:30.570387 12873 raft_consensus.cc:2454] T 1202abbb4862475c8a1085ff59b0821d P e0ae013d6387489c877ad2e47c5e2340 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 874a74185e9d40d88f543d88ba3e4d16 in term 1.
I20260809 22:57:30.570681 12560 leader_election.cc:302] T 1202abbb4862475c8a1085ff59b0821d P 874a74185e9d40d88f543d88ba3e4d16 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2b636635d7764b9a84546f7844221dde, 874a74185e9d40d88f543d88ba3e4d16; no voters:
I20260809 22:57:30.570935 13173 raft_consensus.cc:2781] T 1202abbb4862475c8a1085ff59b0821d P 874a74185e9d40d88f543d88ba3e4d16 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 22:57:30.571359 13173 raft_consensus.cc:686] T 1202abbb4862475c8a1085ff59b0821d P 874a74185e9d40d88f543d88ba3e4d16 [term 1 LEADER]: Becoming Leader. State: Replica: 874a74185e9d40d88f543d88ba3e4d16, State: Running, Role: LEADER
I20260809 22:57:30.571794 13173 consensus_queue.cc:237] T 1202abbb4862475c8a1085ff59b0821d P 874a74185e9d40d88f543d88ba3e4d16 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b636635d7764b9a84546f7844221dde" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 45137 } } peers { permanent_uuid: "874a74185e9d40d88f543d88ba3e4d16" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 42079 } } peers { permanent_uuid: "e0ae013d6387489c877ad2e47c5e2340" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 37919 } }
W20260809 22:57:30.574616 12736 tablet.cc:2369] T 1202abbb4862475c8a1085ff59b0821d P 874a74185e9d40d88f543d88ba3e4d16: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 22:57:30.576521 12444 catalog_manager.cc:5495] T 1202abbb4862475c8a1085ff59b0821d P 874a74185e9d40d88f543d88ba3e4d16 reported cstate change: term changed from 0 to 1, leader changed from <none> to 874a74185e9d40d88f543d88ba3e4d16 (127.15.113.129). New cstate: current_term: 1 leader_uuid: "874a74185e9d40d88f543d88ba3e4d16" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b636635d7764b9a84546f7844221dde" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 45137 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "874a74185e9d40d88f543d88ba3e4d16" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 42079 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e0ae013d6387489c877ad2e47c5e2340" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 37919 } health_report { overall_health: UNKNOWN } } }
I20260809 22:57:30.611853 13162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:57:30.606985 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42372 (local address 127.15.113.129:42079)
0809 22:57:30.607139 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0809 22:57:30.607143 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 22:57:30.607247 (+ 104us) server_negotiation.cc:407] Connection header received
0809 22:57:30.607411 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:57:30.607415 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:57:30.607513 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:57:30.607632 (+ 119us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 22:57:30.607932 (+ 300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:57:30.608609 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:57:30.609973 (+ 1364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:57:30.610412 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:57:30.610500 (+ 88us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:57:30.610930 (+ 430us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 22:57:30.611422 (+ 492us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 22:57:30.611546 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0809 22:57:30.611620 (+ 74us) server_negotiation.cc:299] Negotiation successful
0809 22:57:30.611655 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260809 22:57:30.619635 13073 raft_consensus.cc:1261] T 1202abbb4862475c8a1085ff59b0821d P 2b636635d7764b9a84546f7844221dde [term 1 FOLLOWER]: Refusing update from remote peer 874a74185e9d40d88f543d88ba3e4d16: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 22:57:30.619634 12873 raft_consensus.cc:1261] T 1202abbb4862475c8a1085ff59b0821d P e0ae013d6387489c877ad2e47c5e2340 [term 1 FOLLOWER]: Refusing update from remote peer 874a74185e9d40d88f543d88ba3e4d16: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 22:57:30.620527 13179 consensus_queue.cc:1035] T 1202abbb4862475c8a1085ff59b0821d P 874a74185e9d40d88f543d88ba3e4d16 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e0ae013d6387489c877ad2e47c5e2340" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 37919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 22:57:30.620837 13173 consensus_queue.cc:1035] T 1202abbb4862475c8a1085ff59b0821d P 874a74185e9d40d88f543d88ba3e4d16 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2b636635d7764b9a84546f7844221dde" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 45137 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 22:57:30.650270 12443 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:49710:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 22:57:30.650566 12443 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:49710:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 22:57:30.654057 12443 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5565421b17d949bbbd252a4e92921137: Sending DeleteTablet for 3 replicas of tablet 1202abbb4862475c8a1085ff59b0821d
I20260809 22:57:30.655187 13038 tablet_service.cc:1514] Processing DeleteTablet for tablet 1202abbb4862475c8a1085ff59b0821d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:57:30 UTC) from {username='slave'} at 127.0.0.1:47620
I20260809 22:57:30.655315 12840 tablet_service.cc:1514] Processing DeleteTablet for tablet 1202abbb4862475c8a1085ff59b0821d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:57:30 UTC) from {username='slave'} at 127.0.0.1:53390
I20260809 22:57:30.656087 13205 tablet_replica.cc:330] T 1202abbb4862475c8a1085ff59b0821d P 2b636635d7764b9a84546f7844221dde: stopping tablet replica
I20260809 22:57:30.656114 12630 tablet_service.cc:1514] Processing DeleteTablet for tablet 1202abbb4862475c8a1085ff59b0821d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:57:30 UTC) from {username='slave'} at 127.0.0.1:42368
I20260809 22:57:30.656571 13205 raft_consensus.cc:2229] T 1202abbb4862475c8a1085ff59b0821d P 2b636635d7764b9a84546f7844221dde [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 22:57:30.656630 15814 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskItYPJR/build/asan/bin/kudu pid 12538
I20260809 22:57:30.657197 13205 raft_consensus.cc:2258] T 1202abbb4862475c8a1085ff59b0821d P 2b636635d7764b9a84546f7844221dde [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 22:57:30.659320 13205 ts_tablet_manager.cc:1905] T 1202abbb4862475c8a1085ff59b0821d P 2b636635d7764b9a84546f7844221dde: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 22:57:30.666314 13205 ts_tablet_manager.cc:1918] T 1202abbb4862475c8a1085ff59b0821d P 2b636635d7764b9a84546f7844221dde: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 22:57:30.666529 13205 log.cc:1196] T 1202abbb4862475c8a1085ff59b0821d P 2b636635d7764b9a84546f7844221dde: Deleting WAL directory at /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786315935617919-15814-0/minicluster-data/ts-2/wal/wals/1202abbb4862475c8a1085ff59b0821d
I20260809 22:57:30.667143 13205 ts_tablet_manager.cc:1939] T 1202abbb4862475c8a1085ff59b0821d P 2b636635d7764b9a84546f7844221dde: Deleting consensus metadata
I20260809 22:57:30.667215 13207 tablet_replica.cc:330] T 1202abbb4862475c8a1085ff59b0821d P 874a74185e9d40d88f543d88ba3e4d16: stopping tablet replica
I20260809 22:57:30.667783 13207 raft_consensus.cc:2229] T 1202abbb4862475c8a1085ff59b0821d P 874a74185e9d40d88f543d88ba3e4d16 [term 1 LEADER]: Raft consensus shutting down.
I20260809 22:57:30.668541 13207 raft_consensus.cc:2258] T 1202abbb4862475c8a1085ff59b0821d P 874a74185e9d40d88f543d88ba3e4d16 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 22:57:30.669000 12422 catalog_manager.cc:4841] TS 2b636635d7764b9a84546f7844221dde (127.15.113.131:45137): tablet 1202abbb4862475c8a1085ff59b0821d (table test-table [id=ab1fef4477bd4569bce8ed5657cc81d4]) successfully deleted
I20260809 22:57:30.670867 13207 ts_tablet_manager.cc:1905] T 1202abbb4862475c8a1085ff59b0821d P 874a74185e9d40d88f543d88ba3e4d16: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 22:57:30.674065 13206 tablet_replica.cc:330] T 1202abbb4862475c8a1085ff59b0821d P e0ae013d6387489c877ad2e47c5e2340: stopping tablet replica
I20260809 22:57:30.674537 13206 raft_consensus.cc:2229] T 1202abbb4862475c8a1085ff59b0821d P e0ae013d6387489c877ad2e47c5e2340 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 22:57:30.675087 13206 raft_consensus.cc:2258] T 1202abbb4862475c8a1085ff59b0821d P e0ae013d6387489c877ad2e47c5e2340 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 22:57:30.677372 13206 ts_tablet_manager.cc:1905] T 1202abbb4862475c8a1085ff59b0821d P e0ae013d6387489c877ad2e47c5e2340: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 22:57:30.678611 13207 ts_tablet_manager.cc:1918] T 1202abbb4862475c8a1085ff59b0821d P 874a74185e9d40d88f543d88ba3e4d16: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 22:57:30.678859 13207 log.cc:1196] T 1202abbb4862475c8a1085ff59b0821d P 874a74185e9d40d88f543d88ba3e4d16: Deleting WAL directory at /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786315935617919-15814-0/minicluster-data/ts-0/wal/wals/1202abbb4862475c8a1085ff59b0821d
I20260809 22:57:30.679757 13207 ts_tablet_manager.cc:1939] T 1202abbb4862475c8a1085ff59b0821d P 874a74185e9d40d88f543d88ba3e4d16: Deleting consensus metadata
I20260809 22:57:30.681183 12422 catalog_manager.cc:4841] TS 874a74185e9d40d88f543d88ba3e4d16 (127.15.113.129:42079): tablet 1202abbb4862475c8a1085ff59b0821d (table test-table [id=ab1fef4477bd4569bce8ed5657cc81d4]) successfully deleted
I20260809 22:57:30.684101 13206 ts_tablet_manager.cc:1918] T 1202abbb4862475c8a1085ff59b0821d P e0ae013d6387489c877ad2e47c5e2340: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 22:57:30.684338 13206 log.cc:1196] T 1202abbb4862475c8a1085ff59b0821d P e0ae013d6387489c877ad2e47c5e2340: Deleting WAL directory at /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786315935617919-15814-0/minicluster-data/ts-1/wal/wals/1202abbb4862475c8a1085ff59b0821d
I20260809 22:57:30.684633 13162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:57:30.659170 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42382 (local address 127.15.113.129:42079)
0809 22:57:30.659346 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0809 22:57:30.659353 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:57:30.673413 (+ 14060us) server_negotiation.cc:407] Connection header received
0809 22:57:30.673630 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:57:30.673636 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:57:30.673770 (+ 134us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:57:30.673908 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:57:30.674603 (+ 695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:57:30.675793 (+ 1190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:57:30.677792 (+ 1999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:57:30.678427 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:57:30.678521 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:57:30.683287 (+ 4766us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:57:30.683295 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:57:30.683308 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 22:57:30.683865 (+ 557us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:57:30.683910 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 22:57:30.683929 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:57:30.684020 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0809 22:57:30.684171 (+ 151us) server_negotiation.cc:299] Negotiation successful
0809 22:57:30.684280 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260809 22:57:30.684962 13206 ts_tablet_manager.cc:1939] T 1202abbb4862475c8a1085ff59b0821d P e0ae013d6387489c877ad2e47c5e2340: Deleting consensus metadata
I20260809 22:57:30.685402 12725 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:57:30.686507 12425 catalog_manager.cc:4841] TS e0ae013d6387489c877ad2e47c5e2340 (127.15.113.130:37919): tablet 1202abbb4862475c8a1085ff59b0821d (table test-table [id=ab1fef4477bd4569bce8ed5657cc81d4]) successfully deleted
I20260809 22:57:30.785966 15814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskItYPJR/build/asan/bin/kudu with pid 12538
I20260809 22:57:30.811702 15814 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskItYPJR/build/asan/bin/kudu pid 12745
I20260809 22:57:30.818399 13161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:57:30.812402 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53398 (local address 127.15.113.130:37919)
0809 22:57:30.812567 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0809 22:57:30.812574 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:57:30.812637 (+ 63us) server_negotiation.cc:407] Connection header received
0809 22:57:30.812829 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:57:30.812834 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:57:30.812964 (+ 130us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:57:30.813054 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:57:30.813467 (+ 413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:57:30.814222 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:57:30.815503 (+ 1281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:57:30.815912 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:57:30.816015 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:57:30.817100 (+ 1085us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:57:30.817109 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:57:30.817123 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 22:57:30.817682 (+ 559us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:57:30.817730 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 22:57:30.817757 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:57:30.817853 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0809 22:57:30.817986 (+ 133us) server_negotiation.cc:299] Negotiation successful
0809 22:57:30.818089 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260809 22:57:30.818991 12928 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:57:30.905748 15814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskItYPJR/build/asan/bin/kudu with pid 12745
I20260809 22:57:30.931223 15814 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskItYPJR/build/asan/bin/kudu pid 12939
I20260809 22:57:30.938046 13158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:57:30.931970 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47636 (local address 127.15.113.131:45137)
0809 22:57:30.932159 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0809 22:57:30.932164 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:57:30.932185 (+ 21us) server_negotiation.cc:407] Connection header received
0809 22:57:30.932363 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:57:30.932367 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:57:30.932470 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:57:30.932569 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:57:30.933070 (+ 501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:57:30.933823 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:57:30.935132 (+ 1309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:57:30.935518 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:57:30.935625 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:57:30.936684 (+ 1059us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:57:30.936694 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:57:30.936711 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 22:57:30.937394 (+ 683us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:57:30.937445 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 22:57:30.937474 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:57:30.937561 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0809 22:57:30.937683 (+ 122us) server_negotiation.cc:299] Negotiation successful
0809 22:57:30.937773 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260809 22:57:30.938663 13133 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:57:31.024334 15814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskItYPJR/build/asan/bin/kudu with pid 12939
I20260809 22:57:31.048362 15814 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskItYPJR/build/asan/bin/kudu pid 12401
I20260809 22:57:31.049427 12484 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:57:31.155433 15814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskItYPJR/build/asan/bin/kudu with pid 12401
2026-08-09T22:57:31Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (2889 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315935617919-15814-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 22:57:31 dist-test-slave-ncp9 krb5kdc[13218](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 22:57:31 dist-test-slave-ncp9 krb5kdc[13218](info): set up 2 sockets
Aug 09 22:57:31 dist-test-slave-ncp9 krb5kdc[13218](info): commencing operation
krb5kdc: starting...
W20260809 22:57:32.370826 15814 mini_kdc.cc:121] Time spent starting KDC: real 1.169s user 0.015s sys 0.025s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315935617919-15814-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315935617919-15814-0/krb5kdc/test-user.keytab.
Aug 09 22:57:32 dist-test-slave-ncp9 krb5kdc[13218](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316252, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T22:57:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T22:57:32Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.113.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.113.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.113.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.190.keytab.
Entry for principal kudu/127.15.113.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.190.keytab.
I20260809 22:57:32.658119 15814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskItYPJR/build/asan/bin/kudu
/tmp/dist-test-taskItYPJR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315935617919-15814-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315935617919-15814-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315935617919-15814-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315935617919-15814-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.113.190:42967
--webserver_interface=127.15.113.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.190.keytab
--principal=kudu/127.15.113.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.113.148:39283
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.113.190:42967
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315935617919-15814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315935617919-15814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315935617919-15814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:57:32.985965 13240 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:57:32.986264 13240 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:57:32.986398 13240 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:57:32.993472 13240 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 22:57:32.993598 13240 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:57:32.993680 13240 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:57:32.993758 13240 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 22:57:32.993820 13240 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 22:57:33.003495 13240 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.113.148:39283
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315935617919-15814-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315935617919-15814-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.113.190:42967
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.190.keytab
--principal=kudu/127.15.113.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.113.190:42967
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315935617919-15814-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.113.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315935617919-15814-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:57:33.004779 13240 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:57:33.006147 13240 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:57:33.017674 13245 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:57:33.017642 13250 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:57:33.017709 13246 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:57:33.018462 13240 server_base.cc:1034] running on GCE node
I20260809 22:57:33.019182 13240 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:57:33.020439 13240 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:57:33.021672 13240 hybrid_clock.cc:648] HybridClock initialized: now 1786316253021621 us; error 33 us; skew 500 ppm
I20260809 22:57:33.021997 13240 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:57:33 dist-test-slave-ncp9 krb5kdc[13218](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316253, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.113.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:57:33.024832 13240 init.cc:377] Logged in from keytab as kudu/127.15.113.190@KRBTEST.COM (short username kudu)
I20260809 22:57:33.026285 13240 webserver.cc:460] Webserver started at http://127.15.113.190:42061/ using document root <none> and password file <none>
I20260809 22:57:33.026907 13240 fs_manager.cc:359] Metadata directory not provided
I20260809 22:57:33.027022 13240 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:57:33.027314 13240 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:57:33.030200 13240 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315935617919-15814-0/minicluster-data/master-0/data/instance:
uuid: "2c2ee433cb6e4a1097ab638fc976a454"
format_stamp: "Formatted at 2026-08-09 22:57:33 on dist-test-slave-ncp9"
server_key: "bca78f69288da2288a9cb8ea36272f80"
server_key_iv: "e27eb9b6892de63a4cbec42d3d0e01ad"
server_key_version: "encryptionkey@0"
I20260809 22:57:33.030916 13240 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315935617919-15814-0/minicluster-data/master-0/wal/instance:
uuid: "2c2ee433cb6e4a1097ab638fc976a454"
format_stamp: "Formatted at 2026-08-09 22:57:33 on dist-test-slave-ncp9"
server_key: "bca78f69288da2288a9cb8ea36272f80"
server_key_iv: "e27eb9b6892de63a4cbec42d3d0e01ad"
server_key_version: "encryptionkey@0"
I20260809 22:57:33.035768 13240 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.001s
I20260809 22:57:33.038961 13260 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:57:33.040170 13240 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 22:57:33.040316 13240 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315935617919-15814-0/minicluster-data/master-0/data,/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315935617919-15814-0/minicluster-data/master-0/wal
uuid: "2c2ee433cb6e4a1097ab638fc976a454"
format_stamp: "Formatted at 2026-08-09 22:57:33 on dist-test-slave-ncp9"
server_key: "bca78f69288da2288a9cb8ea36272f80"
server_key_iv: "e27eb9b6892de63a4cbec42d3d0e01ad"
server_key_version: "encryptionkey@0"
I20260809 22:57:33.040481 13240 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315935617919-15814-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315935617919-15814-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315935617919-15814-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:57:33.053960 13240 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:57:33.057488 13240 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:57:33.057731 13240 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:57:33.074647 13240 rpc_server.cc:307] RPC server started. Bound to: 127.15.113.190:42967
I20260809 22:57:33.074666 13340 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.113.190:42967 every 8 connection(s)
I20260809 22:57:33.075888 13240 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315935617919-15814-0/minicluster-data/master-0/data/info.pb
I20260809 22:57:33.079403 13341 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:57:33.083387 15814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskItYPJR/build/asan/bin/kudu as pid 13240
I20260809 22:57:33.083570 15814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315935617919-15814-0/minicluster-data/master-0/wal/instance
I20260809 22:57:33.084088 15814 external_mini_cluster.cc:1428] Setting key 968da54302a78802a0b692c01c0d05aa
I20260809 22:57:33.091533 13341 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2c2ee433cb6e4a1097ab638fc976a454: Bootstrap starting.
I20260809 22:57:33.094411 13341 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2c2ee433cb6e4a1097ab638fc976a454: Neither blocks nor log segments found. Creating new log.
I20260809 22:57:33.095353 13341 log.cc:824] T 00000000000000000000000000000000 P 2c2ee433cb6e4a1097ab638fc976a454: Log is configured to *not* fsync() on all Append() calls
Aug 09 22:57:33 dist-test-slave-ncp9 krb5kdc[13218](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316252, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.113.190@KRBTEST.COM
I20260809 22:57:33.098258 13341 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2c2ee433cb6e4a1097ab638fc976a454: No bootstrap required, opened a new log
I20260809 22:57:33.105969 13345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:57:33.085964 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42750 (local address 127.15.113.190:42967)
0809 22:57:33.086709 (+ 745us) server_negotiation.cc:206] Beginning negotiation
0809 22:57:33.086720 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 22:57:33.086759 (+ 39us) server_negotiation.cc:407] Connection header received
0809 22:57:33.088298 (+ 1539us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:57:33.088319 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:57:33.088741 (+ 422us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:57:33.089120 (+ 379us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:57:33.090614 (+ 1494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:57:33.091927 (+ 1313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:57:33.093263 (+ 1336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:57:33.093908 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:57:33.094064 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:57:33.098878 (+ 4814us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:57:33.098886 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:57:33.098899 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:57:33.098936 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:57:33.101978 (+ 3042us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:57:33.102864 (+ 886us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:57:33.102870 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:57:33.102875 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:57:33.103038 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:57:33.103448 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:57:33.103451 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:57:33.103453 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:57:33.103958 (+ 505us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:57:33.104187 (+ 229us) server_negotiation.cc:1035] Waiting for connection context
0809 22:57:33.104595 (+ 408us) server_negotiation.cc:299] Negotiation successful
0809 22:57:33.104957 (+ 362us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":547,"thread_start_us":337,"threads_started":1}
I20260809 22:57:33.106679 13341 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2c2ee433cb6e4a1097ab638fc976a454 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2c2ee433cb6e4a1097ab638fc976a454" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 42967 } }
I20260809 22:57:33.107009 13341 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2c2ee433cb6e4a1097ab638fc976a454 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:57:33.107182 13341 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2c2ee433cb6e4a1097ab638fc976a454 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c2ee433cb6e4a1097ab638fc976a454, State: Initialized, Role: FOLLOWER
I20260809 22:57:33.107730 13341 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2c2ee433cb6e4a1097ab638fc976a454 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2c2ee433cb6e4a1097ab638fc976a454" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 42967 } }
I20260809 22:57:33.107962 13341 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2c2ee433cb6e4a1097ab638fc976a454 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 22:57:33.108105 13341 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2c2ee433cb6e4a1097ab638fc976a454 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 22:57:33.108265 13341 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2c2ee433cb6e4a1097ab638fc976a454 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:57:33.110427 13341 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2c2ee433cb6e4a1097ab638fc976a454 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2c2ee433cb6e4a1097ab638fc976a454" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 42967 } }
I20260809 22:57:33.110873 13341 leader_election.cc:302] T 00000000000000000000000000000000 P 2c2ee433cb6e4a1097ab638fc976a454 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2c2ee433cb6e4a1097ab638fc976a454; no voters:
I20260809 22:57:33.111413 13341 leader_election.cc:288] T 00000000000000000000000000000000 P 2c2ee433cb6e4a1097ab638fc976a454 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 22:57:33.111572 13349 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2c2ee433cb6e4a1097ab638fc976a454 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 22:57:33.112257 13349 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2c2ee433cb6e4a1097ab638fc976a454 [term 1 LEADER]: Becoming Leader. State: Replica: 2c2ee433cb6e4a1097ab638fc976a454, State: Running, Role: LEADER
I20260809 22:57:33.112759 13349 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2c2ee433cb6e4a1097ab638fc976a454 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2c2ee433cb6e4a1097ab638fc976a454" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 42967 } }
I20260809 22:57:33.113418 13341 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2c2ee433cb6e4a1097ab638fc976a454 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 22:57:33.116385 13352 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2c2ee433cb6e4a1097ab638fc976a454 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2c2ee433cb6e4a1097ab638fc976a454" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2c2ee433cb6e4a1097ab638fc976a454" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 42967 } } }
I20260809 22:57:33.116789 13352 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2c2ee433cb6e4a1097ab638fc976a454 [sys.catalog]: This master's current role is: LEADER
I20260809 22:57:33.117280 13353 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2c2ee433cb6e4a1097ab638fc976a454 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2c2ee433cb6e4a1097ab638fc976a454. Latest consensus state: current_term: 1 leader_uuid: "2c2ee433cb6e4a1097ab638fc976a454" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2c2ee433cb6e4a1097ab638fc976a454" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 42967 } } }
I20260809 22:57:33.117630 13353 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2c2ee433cb6e4a1097ab638fc976a454 [sys.catalog]: This master's current role is: LEADER
I20260809 22:57:33.119992 13364 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 22:57:33.124948 13364 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 22:57:33.133271 13364 catalog_manager.cc:1347] Generated new cluster ID: 4eb30f10112949278e265c4679180a54
I20260809 22:57:33.133421 13364 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 22:57:33.182430 13364 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 22:57:33.184419 13364 catalog_manager.cc:1504] Loading token signing keys...
I20260809 22:57:33.198473 13364 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2c2ee433cb6e4a1097ab638fc976a454: Generated new TSK 0
I20260809 22:57:33.199507 13364 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.113.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.113.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.113.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.129.keytab.
Entry for principal kudu/127.15.113.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.129.keytab.
I20260809 22:57:33.271566 15814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskItYPJR/build/asan/bin/kudu
/tmp/dist-test-taskItYPJR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315935617919-15814-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315935617919-15814-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315935617919-15814-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315935617919-15814-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.113.129:0
--local_ip_for_outbound_sockets=127.15.113.129
--webserver_interface=127.15.113.129
--webserver_port=0
--tserver_master_addrs=127.15.113.190:42967
--keytab_file=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.129.keytab
--principal=kudu/127.15.113.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.113.148:39283
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315935617919-15814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315935617919-15814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315935617919-15814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:57:33.607604 13383 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:57:33.607887 13383 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:57:33.608018 13383 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:57:33.615407 13383 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:57:33.615521 13383 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:57:33.615682 13383 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.113.129
I20260809 22:57:33.625699 13383 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.113.148:39283
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315935617919-15814-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315935617919-15814-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.129.keytab
--principal=kudu/127.15.113.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.113.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315935617919-15814-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.113.129
--webserver_port=0
--tserver_master_addrs=127.15.113.190:42967
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.113.129
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315935617919-15814-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:57:33.627144 13383 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:57:33.628474 13383 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:57:33.641148 13397 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:57:33.641204 13392 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:57:33.641413 13391 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:57:33.642124 13383 server_base.cc:1034] running on GCE node
I20260809 22:57:33.642740 13383 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:57:33.643630 13383 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:57:33.644840 13383 hybrid_clock.cc:648] HybridClock initialized: now 1786316253644789 us; error 42 us; skew 500 ppm
I20260809 22:57:33.645166 13383 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:57:33 dist-test-slave-ncp9 krb5kdc[13218](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316253, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.113.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:57:33.648088 13383 init.cc:377] Logged in from keytab as kudu/127.15.113.129@KRBTEST.COM (short username kudu)
I20260809 22:57:33.649482 13383 webserver.cc:460] Webserver started at http://127.15.113.129:44855/ using document root <none> and password file <none>
I20260809 22:57:33.650110 13383 fs_manager.cc:359] Metadata directory not provided
I20260809 22:57:33.650210 13383 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:57:33.650461 13383 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:57:33.653256 13383 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315935617919-15814-0/minicluster-data/ts-0/data/instance:
uuid: "7b83758d55c04e3382c5fe193529b643"
format_stamp: "Formatted at 2026-08-09 22:57:33 on dist-test-slave-ncp9"
server_key: "bbb3c87fb77921ad944cde0a51c777ba"
server_key_iv: "451b4ce237c2853ec1aa1fc4f70b41b5"
server_key_version: "encryptionkey@0"
I20260809 22:57:33.653975 13383 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315935617919-15814-0/minicluster-data/ts-0/wal/instance:
uuid: "7b83758d55c04e3382c5fe193529b643"
format_stamp: "Formatted at 2026-08-09 22:57:33 on dist-test-slave-ncp9"
server_key: "bbb3c87fb77921ad944cde0a51c777ba"
server_key_iv: "451b4ce237c2853ec1aa1fc4f70b41b5"
server_key_version: "encryptionkey@0"
I20260809 22:57:33.658766 13383 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.004s
I20260809 22:57:33.661928 13413 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:57:33.663151 13383 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 22:57:33.663290 13383 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315935617919-15814-0/minicluster-data/ts-0/data,/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315935617919-15814-0/minicluster-data/ts-0/wal
uuid: "7b83758d55c04e3382c5fe193529b643"
format_stamp: "Formatted at 2026-08-09 22:57:33 on dist-test-slave-ncp9"
server_key: "bbb3c87fb77921ad944cde0a51c777ba"
server_key_iv: "451b4ce237c2853ec1aa1fc4f70b41b5"
server_key_version: "encryptionkey@0"
I20260809 22:57:33.663467 13383 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315935617919-15814-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315935617919-15814-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315935617919-15814-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:57:33.679188 13383 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:57:33.682955 13383 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:57:33.683223 13383 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:57:33.684144 13383 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:57:33.685513 13383 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:57:33.685603 13383 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:57:33.685734 13383 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:57:33.685791 13383 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:57:33.719687 13383 rpc_server.cc:307] RPC server started. Bound to: 127.15.113.129:38943
I20260809 22:57:33.719707 13578 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.113.129:38943 every 8 connection(s)
I20260809 22:57:33.721096 13383 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315935617919-15814-0/minicluster-data/ts-0/data/info.pb
I20260809 22:57:33.728655 15814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskItYPJR/build/asan/bin/kudu as pid 13383
I20260809 22:57:33.728832 15814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315935617919-15814-0/minicluster-data/ts-0/wal/instance
I20260809 22:57:33.729331 15814 external_mini_cluster.cc:1428] Setting key 9199e2559d530b87be66f4207bed5d90
Aug 09 22:57:33 dist-test-slave-ncp9 krb5kdc[13218](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316253, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.113.129@KRBTEST.COM for kudu/127.15.113.190@KRBTEST.COM
I20260809 22:57:33.740659 13581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:57:33.723494 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.113.129:52249 (local address 127.15.113.190:42967)
0809 22:57:33.723931 (+ 437us) server_negotiation.cc:206] Beginning negotiation
0809 22:57:33.723935 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 22:57:33.724874 (+ 939us) server_negotiation.cc:407] Connection header received
0809 22:57:33.726697 (+ 1823us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:57:33.726703 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:57:33.726778 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:57:33.726871 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:57:33.728373 (+ 1502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:57:33.729215 (+ 842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:57:33.730848 (+ 1633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:57:33.731353 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:57:33.731471 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:57:33.735141 (+ 3670us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:57:33.735146 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:57:33.735150 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:57:33.735184 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:57:33.737556 (+ 2372us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:57:33.738369 (+ 813us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:57:33.738373 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:57:33.738377 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:57:33.738499 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:57:33.738979 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:57:33.738982 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:57:33.738985 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:57:33.739247 (+ 262us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:57:33.739364 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0809 22:57:33.740167 (+ 803us) server_negotiation.cc:299] Negotiation successful
0809 22:57:33.740356 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":341,"thread_start_us":229,"threads_started":1}
I20260809 22:57:33.741972 13583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:57:33.724006 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:42967 (local address 127.15.113.129:52249)
0809 22:57:33.724716 (+ 710us) negotiation.cc:107] Waiting for socket to connect
0809 22:57:33.724761 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0809 22:57:33.726430 (+ 1669us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:57:33.727123 (+ 693us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:57:33.727130 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:57:33.727636 (+ 506us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:57:33.728192 (+ 556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:57:33.728210 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:57:33.729363 (+ 1153us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:57:33.729367 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:57:33.730701 (+ 1334us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:57:33.730714 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:57:33.731575 (+ 861us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:57:33.731582 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:57:33.731808 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:57:33.732409 (+ 601us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:57:33.732431 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:57:33.734926 (+ 2495us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:57:33.737694 (+ 2768us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:57:33.737703 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:57:33.737717 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:57:33.738264 (+ 547us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:57:33.738602 (+ 338us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:57:33.738608 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:57:33.738611 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:57:33.738884 (+ 273us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:57:33.739370 (+ 486us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:57:33.739378 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:57:33.739787 (+ 409us) client_negotiation.cc:769] Sending connection context
0809 22:57:33.740104 (+ 317us) client_negotiation.cc:240] Negotiation successful
0809 22:57:33.740608 (+ 504us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":467,"thread_start_us":285,"threads_started":1}
I20260809 22:57:33.743537 13579 heartbeater.cc:347] Connected to a master server at 127.15.113.190:42967
I20260809 22:57:33.743809 13579 heartbeater.cc:464] Registering TS with master...
I20260809 22:57:33.744540 13579 heartbeater.cc:511] Master 127.15.113.190:42967 requested a full tablet report, sending...
I20260809 22:57:33.746683 13285 ts_manager.cc:194] Registered new tserver with Master: 7b83758d55c04e3382c5fe193529b643 (127.15.113.129:38943)
I20260809 22:57:33.748456 13285 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.113.129@KRBTEST.COM'} at 127.15.113.129:52249
WARNING: no policy specified for kudu/127.15.113.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.113.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.113.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.130.keytab.
Entry for principal kudu/127.15.113.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.130.keytab.
I20260809 22:57:33.795859 15814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskItYPJR/build/asan/bin/kudu
/tmp/dist-test-taskItYPJR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315935617919-15814-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315935617919-15814-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315935617919-15814-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315935617919-15814-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.113.130:0
--local_ip_for_outbound_sockets=127.15.113.130
--webserver_interface=127.15.113.130
--webserver_port=0
--tserver_master_addrs=127.15.113.190:42967
--keytab_file=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.130.keytab
--principal=kudu/127.15.113.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.113.148:39283
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315935617919-15814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315935617919-15814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315935617919-15814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:57:34.135005 13587 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:57:34.135258 13587 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:57:34.135424 13587 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:57:34.142633 13587 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:57:34.142745 13587 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:57:34.142885 13587 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.113.130
I20260809 22:57:34.153230 13587 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.113.148:39283
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315935617919-15814-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315935617919-15814-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.130.keytab
--principal=kudu/127.15.113.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.113.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315935617919-15814-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.113.130
--webserver_port=0
--tserver_master_addrs=127.15.113.190:42967
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.113.130
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315935617919-15814-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:57:34.154407 13587 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:57:34.155723 13587 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:57:34.168030 13595 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:57:34.168061 13597 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:57:34.167984 13599 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:57:34.169005 13587 server_base.cc:1034] running on GCE node
I20260809 22:57:34.169543 13587 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:57:34.170387 13587 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:57:34.171576 13587 hybrid_clock.cc:648] HybridClock initialized: now 1786316254171526 us; error 43 us; skew 500 ppm
I20260809 22:57:34.171861 13587 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:57:34 dist-test-slave-ncp9 krb5kdc[13218](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316254, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.113.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:57:34.174837 13587 init.cc:377] Logged in from keytab as kudu/127.15.113.130@KRBTEST.COM (short username kudu)
I20260809 22:57:34.176193 13587 webserver.cc:460] Webserver started at http://127.15.113.130:37081/ using document root <none> and password file <none>
I20260809 22:57:34.176832 13587 fs_manager.cc:359] Metadata directory not provided
I20260809 22:57:34.176988 13587 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:57:34.177285 13587 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:57:34.180028 13587 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315935617919-15814-0/minicluster-data/ts-1/data/instance:
uuid: "1d1ba9f2efa6420aae2f3520464ba981"
format_stamp: "Formatted at 2026-08-09 22:57:34 on dist-test-slave-ncp9"
server_key: "1307ac399e7fd0e684b27175fb723dff"
server_key_iv: "877a0c434032695b4ef3e13366424f6c"
server_key_version: "encryptionkey@0"
I20260809 22:57:34.180771 13587 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315935617919-15814-0/minicluster-data/ts-1/wal/instance:
uuid: "1d1ba9f2efa6420aae2f3520464ba981"
format_stamp: "Formatted at 2026-08-09 22:57:34 on dist-test-slave-ncp9"
server_key: "1307ac399e7fd0e684b27175fb723dff"
server_key_iv: "877a0c434032695b4ef3e13366424f6c"
server_key_version: "encryptionkey@0"
I20260809 22:57:34.185650 13587 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.001s
I20260809 22:57:34.188803 13606 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:57:34.190029 13587 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260809 22:57:34.190169 13587 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315935617919-15814-0/minicluster-data/ts-1/data,/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315935617919-15814-0/minicluster-data/ts-1/wal
uuid: "1d1ba9f2efa6420aae2f3520464ba981"
format_stamp: "Formatted at 2026-08-09 22:57:34 on dist-test-slave-ncp9"
server_key: "1307ac399e7fd0e684b27175fb723dff"
server_key_iv: "877a0c434032695b4ef3e13366424f6c"
server_key_version: "encryptionkey@0"
I20260809 22:57:34.190338 13587 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315935617919-15814-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315935617919-15814-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315935617919-15814-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:57:34.213340 13587 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:57:34.217347 13587 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:57:34.217597 13587 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:57:34.218550 13587 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:57:34.219846 13587 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:57:34.219923 13587 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:57:34.220038 13587 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:57:34.220093 13587 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:57:34.249984 13587 rpc_server.cc:307] RPC server started. Bound to: 127.15.113.130:33049
I20260809 22:57:34.250032 13771 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.113.130:33049 every 8 connection(s)
I20260809 22:57:34.251243 13587 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315935617919-15814-0/minicluster-data/ts-1/data/info.pb
I20260809 22:57:34.253547 15814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskItYPJR/build/asan/bin/kudu as pid 13587
I20260809 22:57:34.253759 15814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315935617919-15814-0/minicluster-data/ts-1/wal/instance
I20260809 22:57:34.254395 15814 external_mini_cluster.cc:1428] Setting key 392d8613b455faccae985b5fd15817d5
Aug 09 22:57:34 dist-test-slave-ncp9 krb5kdc[13218](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316254, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.113.130@KRBTEST.COM for kudu/127.15.113.190@KRBTEST.COM
I20260809 22:57:34.271135 13776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:57:34.253621 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.113.130:33567 (local address 127.15.113.190:42967)
0809 22:57:34.254118 (+ 497us) server_negotiation.cc:206] Beginning negotiation
0809 22:57:34.254122 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 22:57:34.255301 (+ 1179us) server_negotiation.cc:407] Connection header received
0809 22:57:34.257476 (+ 2175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:57:34.257481 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:57:34.257560 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:57:34.257653 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:57:34.259124 (+ 1471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:57:34.259872 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:57:34.261451 (+ 1579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:57:34.261875 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:57:34.261983 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:57:34.265466 (+ 3483us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:57:34.265471 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:57:34.265474 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:57:34.265506 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:57:34.268084 (+ 2578us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:57:34.268866 (+ 782us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:57:34.268870 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:57:34.268872 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:57:34.269020 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:57:34.269438 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:57:34.269441 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:57:34.269443 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:57:34.269701 (+ 258us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:57:34.269808 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0809 22:57:34.270554 (+ 746us) server_negotiation.cc:299] Negotiation successful
0809 22:57:34.270771 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":397,"thread_start_us":273,"threads_started":1}
I20260809 22:57:34.271952 13778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:57:34.254313 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:42967 (local address 127.15.113.130:33567)
0809 22:57:34.255124 (+ 811us) negotiation.cc:107] Waiting for socket to connect
0809 22:57:34.255183 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0809 22:57:34.257213 (+ 2030us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:57:34.257896 (+ 683us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:57:34.257904 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:57:34.258409 (+ 505us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:57:34.258963 (+ 554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:57:34.258981 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:57:34.260022 (+ 1041us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:57:34.260026 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:57:34.261319 (+ 1293us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:57:34.261332 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:57:34.262086 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:57:34.262094 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:57:34.262325 (+ 231us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:57:34.262928 (+ 603us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:57:34.262951 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:57:34.265253 (+ 2302us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:57:34.268205 (+ 2952us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:57:34.268215 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:57:34.268231 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:57:34.268768 (+ 537us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:57:34.269113 (+ 345us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:57:34.269117 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:57:34.269120 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:57:34.269347 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:57:34.269816 (+ 469us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:57:34.269823 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:57:34.270203 (+ 380us) client_negotiation.cc:769] Sending connection context
0809 22:57:34.270478 (+ 275us) client_negotiation.cc:240] Negotiation successful
0809 22:57:34.270899 (+ 421us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":525,"thread_start_us":311,"threads_started":1}
I20260809 22:57:34.273294 13773 heartbeater.cc:347] Connected to a master server at 127.15.113.190:42967
I20260809 22:57:34.273597 13773 heartbeater.cc:464] Registering TS with master...
I20260809 22:57:34.274444 13773 heartbeater.cc:511] Master 127.15.113.190:42967 requested a full tablet report, sending...
I20260809 22:57:34.275946 13285 ts_manager.cc:194] Registered new tserver with Master: 1d1ba9f2efa6420aae2f3520464ba981 (127.15.113.130:33049)
I20260809 22:57:34.276719 13285 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.113.130@KRBTEST.COM'} at 127.15.113.130:33567
WARNING: no policy specified for kudu/127.15.113.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.113.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.113.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.131.keytab.
Entry for principal kudu/127.15.113.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.131.keytab.
I20260809 22:57:34.320186 15814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskItYPJR/build/asan/bin/kudu
/tmp/dist-test-taskItYPJR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315935617919-15814-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315935617919-15814-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315935617919-15814-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315935617919-15814-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.113.131:0
--local_ip_for_outbound_sockets=127.15.113.131
--webserver_interface=127.15.113.131
--webserver_port=0
--tserver_master_addrs=127.15.113.190:42967
--keytab_file=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.131.keytab
--principal=kudu/127.15.113.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.113.148:39283
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315935617919-15814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315935617919-15814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315935617919-15814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:57:34.641840 13786 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:57:34.642118 13786 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:57:34.642263 13786 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:57:34.649283 13786 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:57:34.649377 13786 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:57:34.649492 13786 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.113.131
I20260809 22:57:34.658815 13786 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.113.148:39283
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315935617919-15814-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315935617919-15814-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.131.keytab
--principal=kudu/127.15.113.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.113.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315935617919-15814-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.113.131
--webserver_port=0
--tserver_master_addrs=127.15.113.190:42967
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.113.131
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315935617919-15814-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:57:34.660020 13786 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:57:34.661279 13786 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:57:34.674016 13793 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:57:34.674095 13798 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:57:34.674067 13792 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:57:34.674824 13786 server_base.cc:1034] running on GCE node
I20260809 22:57:34.675360 13786 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:57:34.676208 13786 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:57:34.677399 13786 hybrid_clock.cc:648] HybridClock initialized: now 1786316254677356 us; error 29 us; skew 500 ppm
I20260809 22:57:34.677728 13786 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:57:34 dist-test-slave-ncp9 krb5kdc[13218](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316254, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.113.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:57:34.680634 13786 init.cc:377] Logged in from keytab as kudu/127.15.113.131@KRBTEST.COM (short username kudu)
I20260809 22:57:34.681959 13786 webserver.cc:460] Webserver started at http://127.15.113.131:33245/ using document root <none> and password file <none>
I20260809 22:57:34.682610 13786 fs_manager.cc:359] Metadata directory not provided
I20260809 22:57:34.682729 13786 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:57:34.683060 13786 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:57:34.685913 13786 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315935617919-15814-0/minicluster-data/ts-2/data/instance:
uuid: "c4f27e4b35f84427a07ad95ce2673d33"
format_stamp: "Formatted at 2026-08-09 22:57:34 on dist-test-slave-ncp9"
server_key: "b633db93271b821474cf23cbd4016238"
server_key_iv: "f941ab00039b44ffc6b000397edd71eb"
server_key_version: "encryptionkey@0"
I20260809 22:57:34.686663 13786 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315935617919-15814-0/minicluster-data/ts-2/wal/instance:
uuid: "c4f27e4b35f84427a07ad95ce2673d33"
format_stamp: "Formatted at 2026-08-09 22:57:34 on dist-test-slave-ncp9"
server_key: "b633db93271b821474cf23cbd4016238"
server_key_iv: "f941ab00039b44ffc6b000397edd71eb"
server_key_version: "encryptionkey@0"
I20260809 22:57:34.691558 13786 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260809 22:57:34.694907 13813 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:57:34.696111 13786 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 22:57:34.696250 13786 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315935617919-15814-0/minicluster-data/ts-2/data,/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315935617919-15814-0/minicluster-data/ts-2/wal
uuid: "c4f27e4b35f84427a07ad95ce2673d33"
format_stamp: "Formatted at 2026-08-09 22:57:34 on dist-test-slave-ncp9"
server_key: "b633db93271b821474cf23cbd4016238"
server_key_iv: "f941ab00039b44ffc6b000397edd71eb"
server_key_version: "encryptionkey@0"
I20260809 22:57:34.696412 13786 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315935617919-15814-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315935617919-15814-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315935617919-15814-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:57:34.717849 13786 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:57:34.721833 13786 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:57:34.722080 13786 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:57:34.723022 13786 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:57:34.724275 13786 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:57:34.724346 13786 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:57:34.724434 13786 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:57:34.724476 13786 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:57:34.751729 13579 heartbeater.cc:503] Master 127.15.113.190:42967 was elected leader, sending a full tablet report...
I20260809 22:57:34.753155 13786 rpc_server.cc:307] RPC server started. Bound to: 127.15.113.131:36043
I20260809 22:57:34.753185 13986 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.113.131:36043 every 8 connection(s)
I20260809 22:57:34.754407 13786 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315935617919-15814-0/minicluster-data/ts-2/data/info.pb
I20260809 22:57:34.763972 15814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskItYPJR/build/asan/bin/kudu as pid 13786
I20260809 22:57:34.764146 15814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315935617919-15814-0/minicluster-data/ts-2/wal/instance
I20260809 22:57:34.764564 15814 external_mini_cluster.cc:1428] Setting key 9c19f1b90d31a83e5ee509e1fe2b4812
Aug 09 22:57:34 dist-test-slave-ncp9 krb5kdc[13218](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316254, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.113.131@KRBTEST.COM for kudu/127.15.113.190@KRBTEST.COM
I20260809 22:57:34.772821 13776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:57:34.756694 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.113.131:42087 (local address 127.15.113.190:42967)
0809 22:57:34.756845 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0809 22:57:34.756849 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 22:57:34.758089 (+ 1240us) server_negotiation.cc:407] Connection header received
0809 22:57:34.759841 (+ 1752us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:57:34.759847 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:57:34.759922 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:57:34.760006 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:57:34.761508 (+ 1502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:57:34.762268 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:57:34.763762 (+ 1494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:57:34.764193 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:57:34.764314 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:57:34.768109 (+ 3795us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:57:34.768114 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:57:34.768117 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:57:34.768159 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:57:34.770422 (+ 2263us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:57:34.771038 (+ 616us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:57:34.771041 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:57:34.771044 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:57:34.771162 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:57:34.771477 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:57:34.771480 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:57:34.771482 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:57:34.771731 (+ 249us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:57:34.771844 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0809 22:57:34.772363 (+ 519us) server_negotiation.cc:299] Negotiation successful
0809 22:57:34.772528 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260809 22:57:34.773685 13992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:57:34.757250 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:42967 (local address 127.15.113.131:42087)
0809 22:57:34.757935 (+ 685us) negotiation.cc:107] Waiting for socket to connect
0809 22:57:34.757981 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0809 22:57:34.759590 (+ 1609us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:57:34.760256 (+ 666us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:57:34.760263 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:57:34.760755 (+ 492us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:57:34.761341 (+ 586us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:57:34.761361 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:57:34.762422 (+ 1061us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:57:34.762426 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:57:34.763623 (+ 1197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:57:34.763636 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:57:34.764411 (+ 775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:57:34.764418 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:57:34.764626 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:57:34.765316 (+ 690us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:57:34.765352 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:57:34.767916 (+ 2564us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:57:34.770544 (+ 2628us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:57:34.770551 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:57:34.770561 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:57:34.770950 (+ 389us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:57:34.771234 (+ 284us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:57:34.771237 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:57:34.771239 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:57:34.771401 (+ 162us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:57:34.771816 (+ 415us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:57:34.771821 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:57:34.772084 (+ 263us) client_negotiation.cc:769] Sending connection context
0809 22:57:34.772283 (+ 199us) client_negotiation.cc:240] Negotiation successful
0809 22:57:34.772617 (+ 334us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":463,"thread_start_us":261,"threads_started":1}
I20260809 22:57:34.774892 13988 heartbeater.cc:347] Connected to a master server at 127.15.113.190:42967
I20260809 22:57:34.775131 13988 heartbeater.cc:464] Registering TS with master...
I20260809 22:57:34.775756 13988 heartbeater.cc:511] Master 127.15.113.190:42967 requested a full tablet report, sending...
I20260809 22:57:34.777004 13285 ts_manager.cc:194] Registered new tserver with Master: c4f27e4b35f84427a07ad95ce2673d33 (127.15.113.131:36043)
I20260809 22:57:34.777817 13285 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.113.131@KRBTEST.COM'} at 127.15.113.131:42087
I20260809 22:57:34.779408 15814 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 22:57:34.794306 13776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:57:34.783245 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42752 (local address 127.15.113.190:42967)
0809 22:57:34.783432 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0809 22:57:34.783438 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:57:34.783757 (+ 319us) server_negotiation.cc:407] Connection header received
0809 22:57:34.783885 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:57:34.783889 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:57:34.783952 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:57:34.784018 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:57:34.785368 (+ 1350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:57:34.786069 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:57:34.787245 (+ 1176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:57:34.787657 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:57:34.787743 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:57:34.789913 (+ 2170us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:57:34.789918 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:57:34.789921 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:57:34.789946 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:57:34.792104 (+ 2158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:57:34.792644 (+ 540us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:57:34.792647 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:57:34.792650 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:57:34.792748 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:57:34.793055 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:57:34.793058 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:57:34.793060 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:57:34.793376 (+ 316us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:57:34.793584 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0809 22:57:34.793876 (+ 292us) server_negotiation.cc:299] Negotiation successful
0809 22:57:34.794023 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260809 22:57:34.797708 13284 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42752:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 22:57:34.804373 13284 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 22:57:34.826824 14008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:57:34.815774 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.130:33049 (local address 127.0.0.1:45266)
0809 22:57:34.817302 (+ 1528us) negotiation.cc:107] Waiting for socket to connect
0809 22:57:34.817319 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 22:57:34.817633 (+ 314us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:57:34.818457 (+ 824us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:57:34.818460 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:57:34.818481 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:57:34.818890 (+ 409us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:57:34.818901 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:57:34.820836 (+ 1935us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:57:34.820841 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:57:34.823644 (+ 2803us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:57:34.823662 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:57:34.825760 (+ 2098us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:57:34.825766 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:57:34.826017 (+ 251us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:57:34.826116 (+ 99us) client_negotiation.cc:769] Sending connection context
0809 22:57:34.826239 (+ 123us) client_negotiation.cc:240] Negotiation successful
0809 22:57:34.826420 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1400,"mutex_wait_us":196,"thread_start_us":263,"threads_started":1}
I20260809 22:57:34.826815 14015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:57:34.816074 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51810 (local address 127.15.113.131:36043)
0809 22:57:34.817537 (+ 1463us) server_negotiation.cc:206] Beginning negotiation
0809 22:57:34.817545 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 22:57:34.817572 (+ 27us) server_negotiation.cc:407] Connection header received
0809 22:57:34.817662 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:57:34.817670 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:57:34.817910 (+ 240us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:57:34.818948 (+ 1038us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:57:34.819283 (+ 335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:57:34.820730 (+ 1447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:57:34.823821 (+ 3091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:57:34.825243 (+ 1422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:57:34.825417 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:57:34.825568 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0809 22:57:34.826381 (+ 813us) server_negotiation.cc:299] Negotiation successful
0809 22:57:34.826537 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1305,"thread_start_us":1535,"threads_started":1}
I20260809 22:57:34.827455 14011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:57:34.815521 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.131:36043 (local address 127.0.0.1:51810)
0809 22:57:34.816411 (+ 890us) negotiation.cc:107] Waiting for socket to connect
0809 22:57:34.816464 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0809 22:57:34.816637 (+ 173us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:57:34.818196 (+ 1559us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:57:34.818203 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:57:34.818230 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:57:34.818707 (+ 477us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:57:34.818720 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:57:34.820858 (+ 2138us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:57:34.820863 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:57:34.823639 (+ 2776us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:57:34.823655 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:57:34.825746 (+ 2091us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:57:34.825753 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:57:34.826016 (+ 263us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:57:34.826116 (+ 100us) client_negotiation.cc:769] Sending connection context
0809 22:57:34.826239 (+ 123us) client_negotiation.cc:240] Negotiation successful
0809 22:57:34.827086 (+ 847us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":645,"mutex_wait_us":194,"thread_start_us":1497,"threads_started":1}
I20260809 22:57:34.827709 14013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:57:34.816047 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45266 (local address 127.15.113.130:33049)
0809 22:57:34.817204 (+ 1157us) server_negotiation.cc:206] Beginning negotiation
0809 22:57:34.817212 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 22:57:34.817510 (+ 298us) server_negotiation.cc:407] Connection header received
0809 22:57:34.817817 (+ 307us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:57:34.817824 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:57:34.818019 (+ 195us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:57:34.818189 (+ 170us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:57:34.819070 (+ 881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:57:34.820680 (+ 1610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:57:34.823821 (+ 3141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:57:34.825242 (+ 1421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:57:34.825395 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:57:34.825568 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0809 22:57:34.827227 (+ 1659us) server_negotiation.cc:299] Negotiation successful
0809 22:57:34.827391 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":993,"thread_start_us":872,"threads_started":1}
I20260809 22:57:34.832120 13886 tablet_service.cc:1467] Processing CreateTablet for tablet 9dea92ac77794362a81b9cf6f25f2257 (DEFAULT_TABLE table=test-table [id=1e16e96c76b0496d80c0ac2b9376b623]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:57:34.832116 13689 tablet_service.cc:1467] Processing CreateTablet for tablet 9dea92ac77794362a81b9cf6f25f2257 (DEFAULT_TABLE table=test-table [id=1e16e96c76b0496d80c0ac2b9376b623]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:57:34.833652 13689 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9dea92ac77794362a81b9cf6f25f2257. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:57:34.833706 14011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:57:34.817437 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.129:38943 (local address 127.0.0.1:53652)
0809 22:57:34.827643 (+ 10206us) negotiation.cc:107] Waiting for socket to connect
0809 22:57:34.827660 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 22:57:34.827753 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:57:34.828830 (+ 1077us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:57:34.828834 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:57:34.828859 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:57:34.829016 (+ 157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:57:34.829023 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:57:34.830422 (+ 1399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:57:34.830427 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:57:34.831663 (+ 1236us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:57:34.831683 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:57:34.833084 (+ 1401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:57:34.833090 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:57:34.833288 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:57:34.833312 (+ 24us) client_negotiation.cc:769] Sending connection context
0809 22:57:34.833364 (+ 52us) client_negotiation.cc:240] Negotiation successful
0809 22:57:34.833448 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10124,"thread_start_us":10762,"threads_started":1}
I20260809 22:57:34.834017 13886 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9dea92ac77794362a81b9cf6f25f2257. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:57:34.834527 14018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:57:34.817808 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53652 (local address 127.15.113.129:38943)
0809 22:57:34.828284 (+ 10476us) server_negotiation.cc:206] Beginning negotiation
0809 22:57:34.828291 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:57:34.828318 (+ 27us) server_negotiation.cc:407] Connection header received
0809 22:57:34.828402 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:57:34.828409 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:57:34.828631 (+ 222us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:57:34.829146 (+ 515us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:57:34.829348 (+ 202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:57:34.830285 (+ 937us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:57:34.831820 (+ 1535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:57:34.832753 (+ 933us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:57:34.833969 (+ 1216us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:57:34.834089 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0809 22:57:34.834163 (+ 74us) server_negotiation.cc:299] Negotiation successful
0809 22:57:34.834281 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10303,"thread_start_us":10144,"threads_started":1}
I20260809 22:57:34.837575 13483 tablet_service.cc:1467] Processing CreateTablet for tablet 9dea92ac77794362a81b9cf6f25f2257 (DEFAULT_TABLE table=test-table [id=1e16e96c76b0496d80c0ac2b9376b623]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:57:34.838747 13483 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9dea92ac77794362a81b9cf6f25f2257. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:57:34.848131 14019 tablet_bootstrap.cc:492] T 9dea92ac77794362a81b9cf6f25f2257 P 1d1ba9f2efa6420aae2f3520464ba981: Bootstrap starting.
I20260809 22:57:34.850023 14020 tablet_bootstrap.cc:492] T 9dea92ac77794362a81b9cf6f25f2257 P c4f27e4b35f84427a07ad95ce2673d33: Bootstrap starting.
I20260809 22:57:34.851933 14019 tablet_bootstrap.cc:654] T 9dea92ac77794362a81b9cf6f25f2257 P 1d1ba9f2efa6420aae2f3520464ba981: Neither blocks nor log segments found. Creating new log.
I20260809 22:57:34.853086 14020 tablet_bootstrap.cc:654] T 9dea92ac77794362a81b9cf6f25f2257 P c4f27e4b35f84427a07ad95ce2673d33: Neither blocks nor log segments found. Creating new log.
I20260809 22:57:34.853276 14019 log.cc:824] T 9dea92ac77794362a81b9cf6f25f2257 P 1d1ba9f2efa6420aae2f3520464ba981: Log is configured to *not* fsync() on all Append() calls
I20260809 22:57:34.854120 14020 log.cc:824] T 9dea92ac77794362a81b9cf6f25f2257 P c4f27e4b35f84427a07ad95ce2673d33: Log is configured to *not* fsync() on all Append() calls
I20260809 22:57:34.854593 14023 tablet_bootstrap.cc:492] T 9dea92ac77794362a81b9cf6f25f2257 P 7b83758d55c04e3382c5fe193529b643: Bootstrap starting.
I20260809 22:57:34.857108 14023 tablet_bootstrap.cc:654] T 9dea92ac77794362a81b9cf6f25f2257 P 7b83758d55c04e3382c5fe193529b643: Neither blocks nor log segments found. Creating new log.
I20260809 22:57:34.857987 14023 log.cc:824] T 9dea92ac77794362a81b9cf6f25f2257 P 7b83758d55c04e3382c5fe193529b643: Log is configured to *not* fsync() on all Append() calls
I20260809 22:57:34.858902 14019 tablet_bootstrap.cc:492] T 9dea92ac77794362a81b9cf6f25f2257 P 1d1ba9f2efa6420aae2f3520464ba981: No bootstrap required, opened a new log
I20260809 22:57:34.859221 14019 ts_tablet_manager.cc:1397] T 9dea92ac77794362a81b9cf6f25f2257 P 1d1ba9f2efa6420aae2f3520464ba981: Time spent bootstrapping tablet: real 0.011s user 0.004s sys 0.004s
I20260809 22:57:34.860255 14020 tablet_bootstrap.cc:492] T 9dea92ac77794362a81b9cf6f25f2257 P c4f27e4b35f84427a07ad95ce2673d33: No bootstrap required, opened a new log
I20260809 22:57:34.860522 14020 ts_tablet_manager.cc:1397] T 9dea92ac77794362a81b9cf6f25f2257 P c4f27e4b35f84427a07ad95ce2673d33: Time spent bootstrapping tablet: real 0.011s user 0.000s sys 0.007s
I20260809 22:57:34.860812 14023 tablet_bootstrap.cc:492] T 9dea92ac77794362a81b9cf6f25f2257 P 7b83758d55c04e3382c5fe193529b643: No bootstrap required, opened a new log
I20260809 22:57:34.861094 14023 ts_tablet_manager.cc:1397] T 9dea92ac77794362a81b9cf6f25f2257 P 7b83758d55c04e3382c5fe193529b643: Time spent bootstrapping tablet: real 0.007s user 0.006s sys 0.000s
I20260809 22:57:34.865952 14019 raft_consensus.cc:348] T 9dea92ac77794362a81b9cf6f25f2257 P 1d1ba9f2efa6420aae2f3520464ba981 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4f27e4b35f84427a07ad95ce2673d33" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 36043 } } peers { permanent_uuid: "1d1ba9f2efa6420aae2f3520464ba981" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 33049 } } peers { permanent_uuid: "7b83758d55c04e3382c5fe193529b643" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 38943 } }
I20260809 22:57:34.866295 14019 raft_consensus.cc:374] T 9dea92ac77794362a81b9cf6f25f2257 P 1d1ba9f2efa6420aae2f3520464ba981 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:57:34.866420 14019 raft_consensus.cc:729] T 9dea92ac77794362a81b9cf6f25f2257 P 1d1ba9f2efa6420aae2f3520464ba981 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1d1ba9f2efa6420aae2f3520464ba981, State: Initialized, Role: FOLLOWER
I20260809 22:57:34.867141 14019 consensus_queue.cc:260] T 9dea92ac77794362a81b9cf6f25f2257 P 1d1ba9f2efa6420aae2f3520464ba981 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4f27e4b35f84427a07ad95ce2673d33" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 36043 } } peers { permanent_uuid: "1d1ba9f2efa6420aae2f3520464ba981" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 33049 } } peers { permanent_uuid: "7b83758d55c04e3382c5fe193529b643" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 38943 } }
I20260809 22:57:34.868460 13773 heartbeater.cc:503] Master 127.15.113.190:42967 was elected leader, sending a full tablet report...
I20260809 22:57:34.869923 14019 ts_tablet_manager.cc:1428] T 9dea92ac77794362a81b9cf6f25f2257 P 1d1ba9f2efa6420aae2f3520464ba981: Time spent starting tablet: real 0.011s user 0.009s sys 0.000s
I20260809 22:57:34.871019 14020 raft_consensus.cc:348] T 9dea92ac77794362a81b9cf6f25f2257 P c4f27e4b35f84427a07ad95ce2673d33 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4f27e4b35f84427a07ad95ce2673d33" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 36043 } } peers { permanent_uuid: "1d1ba9f2efa6420aae2f3520464ba981" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 33049 } } peers { permanent_uuid: "7b83758d55c04e3382c5fe193529b643" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 38943 } }
I20260809 22:57:34.871011 14023 raft_consensus.cc:348] T 9dea92ac77794362a81b9cf6f25f2257 P 7b83758d55c04e3382c5fe193529b643 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4f27e4b35f84427a07ad95ce2673d33" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 36043 } } peers { permanent_uuid: "1d1ba9f2efa6420aae2f3520464ba981" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 33049 } } peers { permanent_uuid: "7b83758d55c04e3382c5fe193529b643" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 38943 } }
I20260809 22:57:34.871417 14023 raft_consensus.cc:374] T 9dea92ac77794362a81b9cf6f25f2257 P 7b83758d55c04e3382c5fe193529b643 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:57:34.871417 14020 raft_consensus.cc:374] T 9dea92ac77794362a81b9cf6f25f2257 P c4f27e4b35f84427a07ad95ce2673d33 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:57:34.871518 14023 raft_consensus.cc:729] T 9dea92ac77794362a81b9cf6f25f2257 P 7b83758d55c04e3382c5fe193529b643 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7b83758d55c04e3382c5fe193529b643, State: Initialized, Role: FOLLOWER
I20260809 22:57:34.871518 14020 raft_consensus.cc:729] T 9dea92ac77794362a81b9cf6f25f2257 P c4f27e4b35f84427a07ad95ce2673d33 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4f27e4b35f84427a07ad95ce2673d33, State: Initialized, Role: FOLLOWER
I20260809 22:57:34.872067 14023 consensus_queue.cc:260] T 9dea92ac77794362a81b9cf6f25f2257 P 7b83758d55c04e3382c5fe193529b643 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4f27e4b35f84427a07ad95ce2673d33" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 36043 } } peers { permanent_uuid: "1d1ba9f2efa6420aae2f3520464ba981" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 33049 } } peers { permanent_uuid: "7b83758d55c04e3382c5fe193529b643" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 38943 } }
I20260809 22:57:34.872067 14020 consensus_queue.cc:260] T 9dea92ac77794362a81b9cf6f25f2257 P c4f27e4b35f84427a07ad95ce2673d33 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4f27e4b35f84427a07ad95ce2673d33" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 36043 } } peers { permanent_uuid: "1d1ba9f2efa6420aae2f3520464ba981" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 33049 } } peers { permanent_uuid: "7b83758d55c04e3382c5fe193529b643" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 38943 } }
I20260809 22:57:34.873741 13988 heartbeater.cc:503] Master 127.15.113.190:42967 was elected leader, sending a full tablet report...
I20260809 22:57:34.873930 14023 ts_tablet_manager.cc:1428] T 9dea92ac77794362a81b9cf6f25f2257 P 7b83758d55c04e3382c5fe193529b643: Time spent starting tablet: real 0.013s user 0.012s sys 0.000s
I20260809 22:57:34.874258 14020 ts_tablet_manager.cc:1428] T 9dea92ac77794362a81b9cf6f25f2257 P c4f27e4b35f84427a07ad95ce2673d33: Time spent starting tablet: real 0.014s user 0.014s sys 0.000s
W20260809 22:57:34.973464 13580 tablet.cc:2369] T 9dea92ac77794362a81b9cf6f25f2257 P 7b83758d55c04e3382c5fe193529b643: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 22:57:34.989662 14029 raft_consensus.cc:482] T 9dea92ac77794362a81b9cf6f25f2257 P 7b83758d55c04e3382c5fe193529b643 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 22:57:34.989928 14029 raft_consensus.cc:504] T 9dea92ac77794362a81b9cf6f25f2257 P 7b83758d55c04e3382c5fe193529b643 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4f27e4b35f84427a07ad95ce2673d33" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 36043 } } peers { permanent_uuid: "1d1ba9f2efa6420aae2f3520464ba981" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 33049 } } peers { permanent_uuid: "7b83758d55c04e3382c5fe193529b643" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 38943 } }
I20260809 22:57:34.991592 14029 leader_election.cc:288] T 9dea92ac77794362a81b9cf6f25f2257 P 7b83758d55c04e3382c5fe193529b643 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c4f27e4b35f84427a07ad95ce2673d33 (127.15.113.131:36043), 1d1ba9f2efa6420aae2f3520464ba981 (127.15.113.130:33049)
I20260809 22:57:34.997992 14013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:57:34.992212 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.113.129:44319 (local address 127.15.113.130:33049)
0809 22:57:34.992404 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0809 22:57:34.992410 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:57:34.992714 (+ 304us) server_negotiation.cc:407] Connection header received
0809 22:57:34.992941 (+ 227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:57:34.992948 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:57:34.993012 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:57:34.993090 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:57:34.993688 (+ 598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:57:34.994501 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:57:34.996316 (+ 1815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:57:34.997101 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:57:34.997248 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:57:34.997325 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0809 22:57:34.997618 (+ 293us) server_negotiation.cc:299] Negotiation successful
0809 22:57:34.997748 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260809 22:57:34.998054 14015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:57:34.992047 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.113.129:48745 (local address 127.15.113.131:36043)
0809 22:57:34.992437 (+ 390us) server_negotiation.cc:206] Beginning negotiation
0809 22:57:34.992444 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:57:34.993029 (+ 585us) server_negotiation.cc:407] Connection header received
0809 22:57:34.993201 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:57:34.993206 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:57:34.993271 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:57:34.993345 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:57:34.993905 (+ 560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:57:34.994728 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:57:34.996409 (+ 1681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:57:34.997200 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:57:34.997338 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:57:34.997409 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0809 22:57:34.997690 (+ 281us) server_negotiation.cc:299] Negotiation successful
0809 22:57:34.997814 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":243}
I20260809 22:57:34.998096 14034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:57:34.991950 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.131:36043 (local address 127.15.113.129:48745)
0809 22:57:34.992929 (+ 979us) negotiation.cc:107] Waiting for socket to connect
0809 22:57:34.992943 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 22:57:34.993066 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:57:34.993398 (+ 332us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:57:34.993402 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:57:34.993429 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:57:34.993765 (+ 336us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:57:34.993774 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:57:34.994856 (+ 1082us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:57:34.994861 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:57:34.996262 (+ 1401us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:57:34.996274 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:57:34.997408 (+ 1134us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:57:34.997415 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:57:34.997540 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:57:34.997562 (+ 22us) client_negotiation.cc:769] Sending connection context
0809 22:57:34.997640 (+ 78us) client_negotiation.cc:240] Negotiation successful
0809 22:57:34.997753 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":856,"thread_start_us":670,"threads_started":1}
I20260809 22:57:34.998140 14031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:57:34.991857 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.130:33049 (local address 127.15.113.129:44319)
0809 22:57:34.992584 (+ 727us) negotiation.cc:107] Waiting for socket to connect
0809 22:57:34.992605 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0809 22:57:34.992773 (+ 168us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:57:34.993126 (+ 353us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:57:34.993131 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:57:34.993169 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:57:34.993540 (+ 371us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:57:34.993550 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:57:34.994638 (+ 1088us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:57:34.994643 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:57:34.996155 (+ 1512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:57:34.996168 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:57:34.997311 (+ 1143us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:57:34.997317 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:57:34.997466 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:57:34.997489 (+ 23us) client_negotiation.cc:769] Sending connection context
0809 22:57:34.997572 (+ 83us) client_negotiation.cc:240] Negotiation successful
0809 22:57:34.997704 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":610,"mutex_wait_us":1,"thread_start_us":433,"threads_started":1}
I20260809 22:57:34.999051 13714 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9dea92ac77794362a81b9cf6f25f2257" candidate_uuid: "7b83758d55c04e3382c5fe193529b643" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1d1ba9f2efa6420aae2f3520464ba981" is_pre_election: true
I20260809 22:57:34.999051 13919 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9dea92ac77794362a81b9cf6f25f2257" candidate_uuid: "7b83758d55c04e3382c5fe193529b643" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4f27e4b35f84427a07ad95ce2673d33" is_pre_election: true
I20260809 22:57:34.999531 13919 raft_consensus.cc:2454] T 9dea92ac77794362a81b9cf6f25f2257 P c4f27e4b35f84427a07ad95ce2673d33 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7b83758d55c04e3382c5fe193529b643 in term 0.
I20260809 22:57:34.999545 13714 raft_consensus.cc:2454] T 9dea92ac77794362a81b9cf6f25f2257 P 1d1ba9f2efa6420aae2f3520464ba981 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7b83758d55c04e3382c5fe193529b643 in term 0.
I20260809 22:57:35.000181 13414 leader_election.cc:302] T 9dea92ac77794362a81b9cf6f25f2257 P 7b83758d55c04e3382c5fe193529b643 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7b83758d55c04e3382c5fe193529b643, c4f27e4b35f84427a07ad95ce2673d33; no voters:
I20260809 22:57:35.000538 14029 raft_consensus.cc:2781] T 9dea92ac77794362a81b9cf6f25f2257 P 7b83758d55c04e3382c5fe193529b643 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 22:57:35.000721 14029 raft_consensus.cc:482] T 9dea92ac77794362a81b9cf6f25f2257 P 7b83758d55c04e3382c5fe193529b643 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 22:57:35.000823 14029 raft_consensus.cc:3037] T 9dea92ac77794362a81b9cf6f25f2257 P 7b83758d55c04e3382c5fe193529b643 [term 0 FOLLOWER]: Advancing to term 1
W20260809 22:57:35.003104 13775 tablet.cc:2369] T 9dea92ac77794362a81b9cf6f25f2257 P 1d1ba9f2efa6420aae2f3520464ba981: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 22:57:35.003170 14029 raft_consensus.cc:504] T 9dea92ac77794362a81b9cf6f25f2257 P 7b83758d55c04e3382c5fe193529b643 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4f27e4b35f84427a07ad95ce2673d33" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 36043 } } peers { permanent_uuid: "1d1ba9f2efa6420aae2f3520464ba981" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 33049 } } peers { permanent_uuid: "7b83758d55c04e3382c5fe193529b643" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 38943 } }
I20260809 22:57:35.003892 14029 leader_election.cc:288] T 9dea92ac77794362a81b9cf6f25f2257 P 7b83758d55c04e3382c5fe193529b643 [CANDIDATE]: Term 1 election: Requested vote from peers c4f27e4b35f84427a07ad95ce2673d33 (127.15.113.131:36043), 1d1ba9f2efa6420aae2f3520464ba981 (127.15.113.130:33049)
I20260809 22:57:35.004223 13919 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9dea92ac77794362a81b9cf6f25f2257" candidate_uuid: "7b83758d55c04e3382c5fe193529b643" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4f27e4b35f84427a07ad95ce2673d33"
I20260809 22:57:35.004364 13714 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9dea92ac77794362a81b9cf6f25f2257" candidate_uuid: "7b83758d55c04e3382c5fe193529b643" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1d1ba9f2efa6420aae2f3520464ba981"
I20260809 22:57:35.004408 13919 raft_consensus.cc:3037] T 9dea92ac77794362a81b9cf6f25f2257 P c4f27e4b35f84427a07ad95ce2673d33 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:57:35.004554 13714 raft_consensus.cc:3037] T 9dea92ac77794362a81b9cf6f25f2257 P 1d1ba9f2efa6420aae2f3520464ba981 [term 0 FOLLOWER]: Advancing to term 1
W20260809 22:57:35.005887 13990 tablet.cc:2369] T 9dea92ac77794362a81b9cf6f25f2257 P c4f27e4b35f84427a07ad95ce2673d33: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 22:57:35.006752 13714 raft_consensus.cc:2454] T 9dea92ac77794362a81b9cf6f25f2257 P 1d1ba9f2efa6420aae2f3520464ba981 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7b83758d55c04e3382c5fe193529b643 in term 1.
I20260809 22:57:35.007047 13919 raft_consensus.cc:2454] T 9dea92ac77794362a81b9cf6f25f2257 P c4f27e4b35f84427a07ad95ce2673d33 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7b83758d55c04e3382c5fe193529b643 in term 1.
I20260809 22:57:35.007138 13417 leader_election.cc:302] T 9dea92ac77794362a81b9cf6f25f2257 P 7b83758d55c04e3382c5fe193529b643 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1d1ba9f2efa6420aae2f3520464ba981, 7b83758d55c04e3382c5fe193529b643; no voters:
I20260809 22:57:35.007412 14029 raft_consensus.cc:2781] T 9dea92ac77794362a81b9cf6f25f2257 P 7b83758d55c04e3382c5fe193529b643 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 22:57:35.007812 14029 raft_consensus.cc:686] T 9dea92ac77794362a81b9cf6f25f2257 P 7b83758d55c04e3382c5fe193529b643 [term 1 LEADER]: Becoming Leader. State: Replica: 7b83758d55c04e3382c5fe193529b643, State: Running, Role: LEADER
I20260809 22:57:35.008230 14029 consensus_queue.cc:237] T 9dea92ac77794362a81b9cf6f25f2257 P 7b83758d55c04e3382c5fe193529b643 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4f27e4b35f84427a07ad95ce2673d33" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 36043 } } peers { permanent_uuid: "1d1ba9f2efa6420aae2f3520464ba981" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 33049 } } peers { permanent_uuid: "7b83758d55c04e3382c5fe193529b643" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 38943 } }
I20260809 22:57:35.012771 13285 catalog_manager.cc:5495] T 9dea92ac77794362a81b9cf6f25f2257 P 7b83758d55c04e3382c5fe193529b643 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7b83758d55c04e3382c5fe193529b643 (127.15.113.129). New cstate: current_term: 1 leader_uuid: "7b83758d55c04e3382c5fe193529b643" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4f27e4b35f84427a07ad95ce2673d33" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 36043 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1d1ba9f2efa6420aae2f3520464ba981" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 33049 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7b83758d55c04e3382c5fe193529b643" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 38943 } health_report { overall_health: HEALTHY } } }
I20260809 22:57:35.073019 14018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:57:35.067815 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53656 (local address 127.15.113.129:38943)
0809 22:57:35.068003 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0809 22:57:35.068009 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:57:35.068028 (+ 19us) server_negotiation.cc:407] Connection header received
0809 22:57:35.068250 (+ 222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:57:35.068254 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:57:35.068352 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:57:35.068481 (+ 129us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 22:57:35.068851 (+ 370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:57:35.069643 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:57:35.070960 (+ 1317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:57:35.071354 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:57:35.071461 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:57:35.071959 (+ 498us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 22:57:35.072446 (+ 487us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 22:57:35.072573 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0809 22:57:35.072684 (+ 111us) server_negotiation.cc:299] Negotiation successful
0809 22:57:35.072762 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260809 22:57:35.081810 13919 raft_consensus.cc:1261] T 9dea92ac77794362a81b9cf6f25f2257 P c4f27e4b35f84427a07ad95ce2673d33 [term 1 FOLLOWER]: Refusing update from remote peer 7b83758d55c04e3382c5fe193529b643: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 22:57:35.081810 13714 raft_consensus.cc:1261] T 9dea92ac77794362a81b9cf6f25f2257 P 1d1ba9f2efa6420aae2f3520464ba981 [term 1 FOLLOWER]: Refusing update from remote peer 7b83758d55c04e3382c5fe193529b643: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 22:57:35.082845 14035 consensus_queue.cc:1035] T 9dea92ac77794362a81b9cf6f25f2257 P 7b83758d55c04e3382c5fe193529b643 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1d1ba9f2efa6420aae2f3520464ba981" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 33049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 22:57:35.083190 14029 consensus_queue.cc:1035] T 9dea92ac77794362a81b9cf6f25f2257 P 7b83758d55c04e3382c5fe193529b643 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c4f27e4b35f84427a07ad95ce2673d33" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 36043 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 22:57:35.096796 14040 mvcc.cc:205] Tried to move back new op lower bound from 7316751380803391488 to 7316751380517277696. Current Snapshot: MvccSnapshot[applied={T|T < 7316751380803391488}]
I20260809 22:57:35.108417 13284 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42752:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 22:57:35.108721 13284 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42752:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 22:57:35.112221 13284 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2c2ee433cb6e4a1097ab638fc976a454: Sending DeleteTablet for 3 replicas of tablet 9dea92ac77794362a81b9cf6f25f2257
I20260809 22:57:35.113686 13483 tablet_service.cc:1514] Processing DeleteTablet for tablet 9dea92ac77794362a81b9cf6f25f2257 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:57:35 UTC) from {username='kudu', principal='kudu/127.15.113.190@KRBTEST.COM'} at 127.0.0.1:53652
I20260809 22:57:35.113768 13689 tablet_service.cc:1514] Processing DeleteTablet for tablet 9dea92ac77794362a81b9cf6f25f2257 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:57:35 UTC) from {username='kudu', principal='kudu/127.15.113.190@KRBTEST.COM'} at 127.0.0.1:45266
I20260809 22:57:35.113768 13886 tablet_service.cc:1514] Processing DeleteTablet for tablet 9dea92ac77794362a81b9cf6f25f2257 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:57:35 UTC) from {username='kudu', principal='kudu/127.15.113.190@KRBTEST.COM'} at 127.0.0.1:51810
I20260809 22:57:35.115415 15814 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskItYPJR/build/asan/bin/kudu pid 13383
I20260809 22:57:35.124917 14063 tablet_replica.cc:330] T 9dea92ac77794362a81b9cf6f25f2257 P 1d1ba9f2efa6420aae2f3520464ba981: stopping tablet replica
I20260809 22:57:35.125502 14063 raft_consensus.cc:2229] T 9dea92ac77794362a81b9cf6f25f2257 P 1d1ba9f2efa6420aae2f3520464ba981 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 22:57:35.126109 14064 tablet_replica.cc:330] T 9dea92ac77794362a81b9cf6f25f2257 P c4f27e4b35f84427a07ad95ce2673d33: stopping tablet replica
I20260809 22:57:35.126509 14064 raft_consensus.cc:2229] T 9dea92ac77794362a81b9cf6f25f2257 P c4f27e4b35f84427a07ad95ce2673d33 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 22:57:35.126109 14063 raft_consensus.cc:2258] T 9dea92ac77794362a81b9cf6f25f2257 P 1d1ba9f2efa6420aae2f3520464ba981 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 22:57:35.127000 14064 raft_consensus.cc:2258] T 9dea92ac77794362a81b9cf6f25f2257 P c4f27e4b35f84427a07ad95ce2673d33 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 22:57:35.127300 14060 tablet_replica.cc:330] T 9dea92ac77794362a81b9cf6f25f2257 P 7b83758d55c04e3382c5fe193529b643: stopping tablet replica
I20260809 22:57:35.127722 14060 raft_consensus.cc:2229] T 9dea92ac77794362a81b9cf6f25f2257 P 7b83758d55c04e3382c5fe193529b643 [term 1 LEADER]: Raft consensus shutting down.
I20260809 22:57:35.128405 14060 raft_consensus.cc:2258] T 9dea92ac77794362a81b9cf6f25f2257 P 7b83758d55c04e3382c5fe193529b643 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 22:57:35.129215 14064 ts_tablet_manager.cc:1905] T 9dea92ac77794362a81b9cf6f25f2257 P c4f27e4b35f84427a07ad95ce2673d33: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 22:57:35.129411 14063 ts_tablet_manager.cc:1905] T 9dea92ac77794362a81b9cf6f25f2257 P 1d1ba9f2efa6420aae2f3520464ba981: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 22:57:35.135174 14060 ts_tablet_manager.cc:1905] T 9dea92ac77794362a81b9cf6f25f2257 P 7b83758d55c04e3382c5fe193529b643: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 22:57:35.136070 14064 ts_tablet_manager.cc:1918] T 9dea92ac77794362a81b9cf6f25f2257 P c4f27e4b35f84427a07ad95ce2673d33: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 22:57:35.136296 14064 log.cc:1196] T 9dea92ac77794362a81b9cf6f25f2257 P c4f27e4b35f84427a07ad95ce2673d33: Deleting WAL directory at /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315935617919-15814-0/minicluster-data/ts-2/wal/wals/9dea92ac77794362a81b9cf6f25f2257
I20260809 22:57:35.136828 14063 ts_tablet_manager.cc:1918] T 9dea92ac77794362a81b9cf6f25f2257 P 1d1ba9f2efa6420aae2f3520464ba981: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 22:57:35.136946 14064 ts_tablet_manager.cc:1939] T 9dea92ac77794362a81b9cf6f25f2257 P c4f27e4b35f84427a07ad95ce2673d33: Deleting consensus metadata
I20260809 22:57:35.137101 14063 log.cc:1196] T 9dea92ac77794362a81b9cf6f25f2257 P 1d1ba9f2efa6420aae2f3520464ba981: Deleting WAL directory at /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315935617919-15814-0/minicluster-data/ts-1/wal/wals/9dea92ac77794362a81b9cf6f25f2257
I20260809 22:57:35.137684 14063 ts_tablet_manager.cc:1939] T 9dea92ac77794362a81b9cf6f25f2257 P 1d1ba9f2efa6420aae2f3520464ba981: Deleting consensus metadata
I20260809 22:57:35.138775 13261 catalog_manager.cc:4841] TS c4f27e4b35f84427a07ad95ce2673d33 (127.15.113.131:36043): tablet 9dea92ac77794362a81b9cf6f25f2257 (table test-table [id=1e16e96c76b0496d80c0ac2b9376b623]) successfully deleted
I20260809 22:57:35.139340 13264 catalog_manager.cc:4841] TS 1d1ba9f2efa6420aae2f3520464ba981 (127.15.113.130:33049): tablet 9dea92ac77794362a81b9cf6f25f2257 (table test-table [id=1e16e96c76b0496d80c0ac2b9376b623]) successfully deleted
I20260809 22:57:35.141687 14060 ts_tablet_manager.cc:1918] T 9dea92ac77794362a81b9cf6f25f2257 P 7b83758d55c04e3382c5fe193529b643: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 22:57:35.141849 14060 log.cc:1196] T 9dea92ac77794362a81b9cf6f25f2257 P 7b83758d55c04e3382c5fe193529b643: Deleting WAL directory at /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315935617919-15814-0/minicluster-data/ts-0/wal/wals/9dea92ac77794362a81b9cf6f25f2257
I20260809 22:57:35.142405 14060 ts_tablet_manager.cc:1939] T 9dea92ac77794362a81b9cf6f25f2257 P 7b83758d55c04e3382c5fe193529b643: Deleting consensus metadata
I20260809 22:57:35.143787 13261 catalog_manager.cc:4841] TS 7b83758d55c04e3382c5fe193529b643 (127.15.113.129:38943): tablet 9dea92ac77794362a81b9cf6f25f2257 (table test-table [id=1e16e96c76b0496d80c0ac2b9376b623]) successfully deleted
Aug 09 22:57:35 dist-test-slave-ncp9 krb5kdc[13218](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316252, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.113.129@KRBTEST.COM
I20260809 22:57:35.154762 14018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:57:35.116690 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53666 (local address 127.15.113.129:38943)
0809 22:57:35.116862 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0809 22:57:35.116868 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:57:35.131675 (+ 14807us) server_negotiation.cc:407] Connection header received
0809 22:57:35.131873 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:57:35.131878 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:57:35.131961 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:57:35.132049 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:57:35.139772 (+ 7723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:57:35.141052 (+ 1280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:57:35.142854 (+ 1802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:57:35.143593 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:57:35.143728 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:57:35.148175 (+ 4447us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:57:35.148189 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:57:35.148207 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:57:35.148251 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:57:35.151983 (+ 3732us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:57:35.152632 (+ 649us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:57:35.152642 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:57:35.152646 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:57:35.152844 (+ 198us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:57:35.153239 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:57:35.153244 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:57:35.153247 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:57:35.153699 (+ 452us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:57:35.153979 (+ 280us) server_negotiation.cc:1035] Waiting for connection context
0809 22:57:35.154159 (+ 180us) server_negotiation.cc:299] Negotiation successful
0809 22:57:35.154391 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260809 22:57:35.155505 13574 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:57:35.253535 15814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskItYPJR/build/asan/bin/kudu with pid 13383
I20260809 22:57:35.283460 15814 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskItYPJR/build/asan/bin/kudu pid 13587
Aug 09 22:57:35 dist-test-slave-ncp9 krb5kdc[13218](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316252, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.113.130@KRBTEST.COM
I20260809 22:57:35.298738 14013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:57:35.284304 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45268 (local address 127.15.113.130:33049)
0809 22:57:35.284509 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0809 22:57:35.284516 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:57:35.284539 (+ 23us) server_negotiation.cc:407] Connection header received
0809 22:57:35.284719 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:57:35.284724 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:57:35.284790 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:57:35.284918 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:57:35.286347 (+ 1429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:57:35.287131 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:57:35.288474 (+ 1343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:57:35.289029 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:57:35.289161 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:57:35.293397 (+ 4236us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:57:35.293410 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:57:35.293424 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:57:35.293462 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:57:35.296249 (+ 2787us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:57:35.296953 (+ 704us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:57:35.296960 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:57:35.296963 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:57:35.297079 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:57:35.297402 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:57:35.297405 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:57:35.297406 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:57:35.297792 (+ 386us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:57:35.298009 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0809 22:57:35.298263 (+ 254us) server_negotiation.cc:299] Negotiation successful
0809 22:57:35.298420 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260809 22:57:35.299463 13765 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:57:35.387519 15814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskItYPJR/build/asan/bin/kudu with pid 13587
I20260809 22:57:35.413656 15814 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskItYPJR/build/asan/bin/kudu pid 13786
Aug 09 22:57:35 dist-test-slave-ncp9 krb5kdc[13218](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316252, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.113.131@KRBTEST.COM
I20260809 22:57:35.429354 14015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:57:35.414455 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51814 (local address 127.15.113.131:36043)
0809 22:57:35.414655 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0809 22:57:35.414660 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:57:35.414724 (+ 64us) server_negotiation.cc:407] Connection header received
0809 22:57:35.414949 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:57:35.414956 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:57:35.415040 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:57:35.415152 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:57:35.416437 (+ 1285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:57:35.417356 (+ 919us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:57:35.418831 (+ 1475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:57:35.419220 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:57:35.419326 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:57:35.423696 (+ 4370us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:57:35.423713 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:57:35.423730 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:57:35.423767 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:57:35.426430 (+ 2663us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:57:35.427170 (+ 740us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:57:35.427179 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:57:35.427181 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:57:35.427324 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:57:35.427712 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:57:35.427718 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:57:35.427720 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:57:35.428120 (+ 400us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:57:35.428413 (+ 293us) server_negotiation.cc:1035] Waiting for connection context
0809 22:57:35.428731 (+ 318us) server_negotiation.cc:299] Negotiation successful
0809 22:57:35.429007 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260809 22:57:35.429989 13980 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:57:35.521674 15814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskItYPJR/build/asan/bin/kudu with pid 13786
I20260809 22:57:35.544636 15814 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskItYPJR/build/asan/bin/kudu pid 13240
I20260809 22:57:35.545740 13332 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:57:35.657246 15814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskItYPJR/build/asan/bin/kudu with pid 13240
2026-08-09T22:57:35Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (4502 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-08-09T22:57:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T22:57:35Z Disabled control of system clock
I20260809 22:57:35.751035 15814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskItYPJR/build/asan/bin/kudu
/tmp/dist-test-taskItYPJR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786315935617919-15814-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786315935617919-15814-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786315935617919-15814-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786315935617919-15814-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.113.190:34403
--webserver_interface=127.15.113.190
--webserver_port=0
--builtin_ntp_servers=127.15.113.148:38275
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.113.190:34403
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 22:57:36.076109 14078 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:57:36.076375 14078 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:57:36.076504 14078 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:57:36.083529 14078 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 22:57:36.083636 14078 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:57:36.083707 14078 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:57:36.083777 14078 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 22:57:36.083835 14078 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 22:57:36.092515 14078 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.113.148:38275
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786315935617919-15814-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786315935617919-15814-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.113.190:34403
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.113.190:34403
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786315935617919-15814-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.113.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786315935617919-15814-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:57:36.093752 14078 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:57:36.095496 14078 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:57:36.106488 14085 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:57:36.106539 14089 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:57:36.106572 14087 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:57:36.107688 14078 server_base.cc:1034] running on GCE node
I20260809 22:57:36.108425 14078 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:57:36.109741 14078 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:57:36.110939 14078 hybrid_clock.cc:648] HybridClock initialized: now 1786316256110901 us; error 30 us; skew 500 ppm
I20260809 22:57:36.111248 14078 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 22:57:36.116214 14078 webserver.cc:460] Webserver started at http://127.15.113.190:36297/ using document root <none> and password file <none>
I20260809 22:57:36.116873 14078 fs_manager.cc:359] Metadata directory not provided
I20260809 22:57:36.117013 14078 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:57:36.117303 14078 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:57:36.120177 14078 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786315935617919-15814-0/minicluster-data/master-0/data/instance:
uuid: "eb4a5fe13a9145288382f79b64f52f2b"
format_stamp: "Formatted at 2026-08-09 22:57:36 on dist-test-slave-ncp9"
server_key: "b04ae8c3cad0a79013e6944308c2d42e"
server_key_iv: "6e3c1c80ac7b25457c7cd9e3d5718c56"
server_key_version: "encryptionkey@0"
I20260809 22:57:36.120954 14078 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786315935617919-15814-0/minicluster-data/master-0/wal/instance:
uuid: "eb4a5fe13a9145288382f79b64f52f2b"
format_stamp: "Formatted at 2026-08-09 22:57:36 on dist-test-slave-ncp9"
server_key: "b04ae8c3cad0a79013e6944308c2d42e"
server_key_iv: "6e3c1c80ac7b25457c7cd9e3d5718c56"
server_key_version: "encryptionkey@0"
I20260809 22:57:36.125777 14078 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.001s
I20260809 22:57:36.129164 14097 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:57:36.130429 14078 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 22:57:36.130585 14078 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786315935617919-15814-0/minicluster-data/master-0/data,/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786315935617919-15814-0/minicluster-data/master-0/wal
uuid: "eb4a5fe13a9145288382f79b64f52f2b"
format_stamp: "Formatted at 2026-08-09 22:57:36 on dist-test-slave-ncp9"
server_key: "b04ae8c3cad0a79013e6944308c2d42e"
server_key_iv: "6e3c1c80ac7b25457c7cd9e3d5718c56"
server_key_version: "encryptionkey@0"
I20260809 22:57:36.130775 14078 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786315935617919-15814-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786315935617919-15814-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786315935617919-15814-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:57:36.144515 14078 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:57:36.145278 14078 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:57:36.145524 14078 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:57:36.162103 14078 rpc_server.cc:307] RPC server started. Bound to: 127.15.113.190:34403
I20260809 22:57:36.162138 14189 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.113.190:34403 every 8 connection(s)
I20260809 22:57:36.163370 14078 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786315935617919-15814-0/minicluster-data/master-0/data/info.pb
I20260809 22:57:36.164835 15814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskItYPJR/build/asan/bin/kudu as pid 14078
I20260809 22:57:36.165012 15814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786315935617919-15814-0/minicluster-data/master-0/wal/instance
I20260809 22:57:36.165442 15814 external_mini_cluster.cc:1428] Setting key 9a60c2e9e0fa8dba39ccbe6922e8fe04
I20260809 22:57:36.167336 14190 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:57:36.180210 14197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:57:36.167210 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35370 (local address 127.15.113.190:34403)
0809 22:57:36.168325 (+ 1115us) server_negotiation.cc:206] Beginning negotiation
0809 22:57:36.168340 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0809 22:57:36.168391 (+ 51us) server_negotiation.cc:407] Connection header received
0809 22:57:36.170427 (+ 2036us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:57:36.170454 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:57:36.171035 (+ 581us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:57:36.171455 (+ 420us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:57:36.172004 (+ 549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:57:36.174001 (+ 1997us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:57:36.175272 (+ 1271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:57:36.176270 (+ 998us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:57:36.176451 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:57:36.177526 (+ 1075us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:57:36.177537 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:57:36.177554 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 22:57:36.178279 (+ 725us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:57:36.178381 (+ 102us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 22:57:36.178407 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:57:36.178529 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0809 22:57:36.178692 (+ 163us) server_negotiation.cc:299] Negotiation successful
0809 22:57:36.179022 (+ 330us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":630,"thread_start_us":401,"threads_started":1}
I20260809 22:57:36.181519 14190 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eb4a5fe13a9145288382f79b64f52f2b: Bootstrap starting.
I20260809 22:57:36.184581 14190 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P eb4a5fe13a9145288382f79b64f52f2b: Neither blocks nor log segments found. Creating new log.
I20260809 22:57:36.185578 14190 log.cc:824] T 00000000000000000000000000000000 P eb4a5fe13a9145288382f79b64f52f2b: Log is configured to *not* fsync() on all Append() calls
I20260809 22:57:36.187991 14190 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eb4a5fe13a9145288382f79b64f52f2b: No bootstrap required, opened a new log
I20260809 22:57:36.194988 14190 raft_consensus.cc:348] T 00000000000000000000000000000000 P eb4a5fe13a9145288382f79b64f52f2b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb4a5fe13a9145288382f79b64f52f2b" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 34403 } }
I20260809 22:57:36.195323 14190 raft_consensus.cc:374] T 00000000000000000000000000000000 P eb4a5fe13a9145288382f79b64f52f2b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:57:36.195417 14190 raft_consensus.cc:729] T 00000000000000000000000000000000 P eb4a5fe13a9145288382f79b64f52f2b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb4a5fe13a9145288382f79b64f52f2b, State: Initialized, Role: FOLLOWER
I20260809 22:57:36.195941 14190 consensus_queue.cc:260] T 00000000000000000000000000000000 P eb4a5fe13a9145288382f79b64f52f2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb4a5fe13a9145288382f79b64f52f2b" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 34403 } }
I20260809 22:57:36.196161 14190 raft_consensus.cc:388] T 00000000000000000000000000000000 P eb4a5fe13a9145288382f79b64f52f2b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 22:57:36.196288 14190 raft_consensus.cc:482] T 00000000000000000000000000000000 P eb4a5fe13a9145288382f79b64f52f2b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 22:57:36.196449 14190 raft_consensus.cc:3037] T 00000000000000000000000000000000 P eb4a5fe13a9145288382f79b64f52f2b [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:57:36.198666 14190 raft_consensus.cc:504] T 00000000000000000000000000000000 P eb4a5fe13a9145288382f79b64f52f2b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb4a5fe13a9145288382f79b64f52f2b" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 34403 } }
I20260809 22:57:36.199139 14190 leader_election.cc:302] T 00000000000000000000000000000000 P eb4a5fe13a9145288382f79b64f52f2b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: eb4a5fe13a9145288382f79b64f52f2b; no voters:
I20260809 22:57:36.199668 14190 leader_election.cc:288] T 00000000000000000000000000000000 P eb4a5fe13a9145288382f79b64f52f2b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 22:57:36.199786 14199 raft_consensus.cc:2781] T 00000000000000000000000000000000 P eb4a5fe13a9145288382f79b64f52f2b [term 1 FOLLOWER]: Leader election won for term 1
I20260809 22:57:36.200487 14199 raft_consensus.cc:686] T 00000000000000000000000000000000 P eb4a5fe13a9145288382f79b64f52f2b [term 1 LEADER]: Becoming Leader. State: Replica: eb4a5fe13a9145288382f79b64f52f2b, State: Running, Role: LEADER
I20260809 22:57:36.201028 14199 consensus_queue.cc:237] T 00000000000000000000000000000000 P eb4a5fe13a9145288382f79b64f52f2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb4a5fe13a9145288382f79b64f52f2b" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 34403 } }
I20260809 22:57:36.201586 14190 sys_catalog.cc:564] T 00000000000000000000000000000000 P eb4a5fe13a9145288382f79b64f52f2b [sys.catalog]: configured and running, proceeding with master startup.
I20260809 22:57:36.203735 14201 sys_catalog.cc:455] T 00000000000000000000000000000000 P eb4a5fe13a9145288382f79b64f52f2b [sys.catalog]: SysCatalogTable state changed. Reason: New leader eb4a5fe13a9145288382f79b64f52f2b. Latest consensus state: current_term: 1 leader_uuid: "eb4a5fe13a9145288382f79b64f52f2b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb4a5fe13a9145288382f79b64f52f2b" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 34403 } } }
I20260809 22:57:36.203877 14200 sys_catalog.cc:455] T 00000000000000000000000000000000 P eb4a5fe13a9145288382f79b64f52f2b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "eb4a5fe13a9145288382f79b64f52f2b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb4a5fe13a9145288382f79b64f52f2b" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 34403 } } }
I20260809 22:57:36.204146 14201 sys_catalog.cc:458] T 00000000000000000000000000000000 P eb4a5fe13a9145288382f79b64f52f2b [sys.catalog]: This master's current role is: LEADER
I20260809 22:57:36.204186 14200 sys_catalog.cc:458] T 00000000000000000000000000000000 P eb4a5fe13a9145288382f79b64f52f2b [sys.catalog]: This master's current role is: LEADER
I20260809 22:57:36.206053 14210 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 22:57:36.210995 14210 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 22:57:36.219887 14210 catalog_manager.cc:1347] Generated new cluster ID: 28a0860f43994e0c8205b6db671f01af
I20260809 22:57:36.219978 14210 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 22:57:36.234968 14210 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 22:57:36.236150 14210 catalog_manager.cc:1504] Loading token signing keys...
I20260809 22:57:36.251660 14210 catalog_manager.cc:5868] T 00000000000000000000000000000000 P eb4a5fe13a9145288382f79b64f52f2b: Generated new TSK 0
I20260809 22:57:36.252413 14210 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 22:57:36.259289 15814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskItYPJR/build/asan/bin/kudu
/tmp/dist-test-taskItYPJR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786315935617919-15814-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786315935617919-15814-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786315935617919-15814-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786315935617919-15814-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.113.129:0
--local_ip_for_outbound_sockets=127.15.113.129
--webserver_interface=127.15.113.129
--webserver_port=0
--tserver_master_addrs=127.15.113.190:34403
--builtin_ntp_servers=127.15.113.148:38275
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 22:57:36.606958 14232 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:57:36.607262 14232 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:57:36.607393 14232 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:57:36.614511 14232 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:57:36.614611 14232 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:57:36.614759 14232 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.113.129
I20260809 22:57:36.623708 14232 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.113.148:38275
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786315935617919-15814-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786315935617919-15814-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.113.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786315935617919-15814-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.113.129
--webserver_port=0
--tserver_master_addrs=127.15.113.190:34403
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.113.129
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786315935617919-15814-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:57:36.624972 14232 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:57:36.626771 14232 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:57:36.639140 14237 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:57:36.639137 14238 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:57:36.639134 14241 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:57:36.640015 14232 server_base.cc:1034] running on GCE node
I20260809 22:57:36.640558 14232 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:57:36.641436 14232 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:57:36.642635 14232 hybrid_clock.cc:648] HybridClock initialized: now 1786316256642584 us; error 37 us; skew 500 ppm
I20260809 22:57:36.642916 14232 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 22:57:36.644948 14232 webserver.cc:460] Webserver started at http://127.15.113.129:45263/ using document root <none> and password file <none>
I20260809 22:57:36.645576 14232 fs_manager.cc:359] Metadata directory not provided
I20260809 22:57:36.645674 14232 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:57:36.645969 14232 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:57:36.648806 14232 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786315935617919-15814-0/minicluster-data/ts-0/data/instance:
uuid: "8c4d0483666a41b7acbf64634756da9e"
format_stamp: "Formatted at 2026-08-09 22:57:36 on dist-test-slave-ncp9"
server_key: "771af37f71cc45b221f7e893068fb51e"
server_key_iv: "a040a5c069e479e83cd71f2fa270848d"
server_key_version: "encryptionkey@0"
I20260809 22:57:36.649597 14232 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786315935617919-15814-0/minicluster-data/ts-0/wal/instance:
uuid: "8c4d0483666a41b7acbf64634756da9e"
format_stamp: "Formatted at 2026-08-09 22:57:36 on dist-test-slave-ncp9"
server_key: "771af37f71cc45b221f7e893068fb51e"
server_key_iv: "a040a5c069e479e83cd71f2fa270848d"
server_key_version: "encryptionkey@0"
I20260809 22:57:36.654243 14232 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.002s
I20260809 22:57:36.657410 14253 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:57:36.658588 14232 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 22:57:36.658730 14232 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786315935617919-15814-0/minicluster-data/ts-0/data,/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786315935617919-15814-0/minicluster-data/ts-0/wal
uuid: "8c4d0483666a41b7acbf64634756da9e"
format_stamp: "Formatted at 2026-08-09 22:57:36 on dist-test-slave-ncp9"
server_key: "771af37f71cc45b221f7e893068fb51e"
server_key_iv: "a040a5c069e479e83cd71f2fa270848d"
server_key_version: "encryptionkey@0"
I20260809 22:57:36.658912 14232 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786315935617919-15814-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786315935617919-15814-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786315935617919-15814-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:57:36.678138 14232 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:57:36.678861 14232 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:57:36.679127 14232 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:57:36.680212 14232 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:57:36.681762 14232 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:57:36.681871 14232 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:57:36.681995 14232 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:57:36.682075 14232 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:57:36.714824 14232 rpc_server.cc:307] RPC server started. Bound to: 127.15.113.129:34679
I20260809 22:57:36.714916 14431 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.113.129:34679 every 8 connection(s)
I20260809 22:57:36.716061 14232 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786315935617919-15814-0/minicluster-data/ts-0/data/info.pb
I20260809 22:57:36.725652 15814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskItYPJR/build/asan/bin/kudu as pid 14232
I20260809 22:57:36.725832 15814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786315935617919-15814-0/minicluster-data/ts-0/wal/instance
I20260809 22:57:36.726346 15814 external_mini_cluster.cc:1428] Setting key 5d30d9555be66f980bddc2b92ca59f34
I20260809 22:57:36.728065 14434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:57:36.718535 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.113.129:56433 (local address 127.15.113.190:34403)
0809 22:57:36.718969 (+ 434us) server_negotiation.cc:206] Beginning negotiation
0809 22:57:36.718974 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:57:36.720142 (+ 1168us) server_negotiation.cc:407] Connection header received
0809 22:57:36.721886 (+ 1744us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:57:36.721891 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:57:36.721958 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:57:36.722053 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:57:36.723040 (+ 987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:57:36.723747 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:57:36.725247 (+ 1500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:57:36.725822 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:57:36.725952 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:57:36.727224 (+ 1272us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:57:36.727230 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:57:36.727233 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 22:57:36.727251 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:57:36.727289 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 22:57:36.727304 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:57:36.727392 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0809 22:57:36.727606 (+ 214us) server_negotiation.cc:299] Negotiation successful
0809 22:57:36.727694 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":333,"thread_start_us":232,"threads_started":1}
I20260809 22:57:36.729151 14435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:57:36.719040 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:34403 (local address 127.15.113.129:56433)
0809 22:57:36.719931 (+ 891us) negotiation.cc:107] Waiting for socket to connect
0809 22:57:36.720005 (+ 74us) client_negotiation.cc:174] Beginning negotiation
0809 22:57:36.721656 (+ 1651us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:57:36.722245 (+ 589us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:57:36.722253 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:57:36.722337 (+ 84us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:57:36.722901 (+ 564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:57:36.722918 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:57:36.723857 (+ 939us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:57:36.723860 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:57:36.725142 (+ 1282us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:57:36.725154 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:57:36.726030 (+ 876us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:57:36.726035 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:57:36.726227 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:57:36.726916 (+ 689us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 22:57:36.726919 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:57:36.726957 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 22:57:36.727037 (+ 80us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:57:36.727408 (+ 371us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:57:36.727418 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:57:36.727424 (+ 6us) client_negotiation.cc:769] Sending connection context
0809 22:57:36.727571 (+ 147us) client_negotiation.cc:240] Negotiation successful
0809 22:57:36.727860 (+ 289us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":455,"thread_start_us":261,"threads_started":1}
I20260809 22:57:36.729503 15814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskItYPJR/build/asan/bin/kudu
/tmp/dist-test-taskItYPJR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786315935617919-15814-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786315935617919-15814-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786315935617919-15814-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786315935617919-15814-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.113.130:0
--local_ip_for_outbound_sockets=127.15.113.130
--webserver_interface=127.15.113.130
--webserver_port=0
--tserver_master_addrs=127.15.113.190:34403
--builtin_ntp_servers=127.15.113.148:38275
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260809 22:57:36.730638 14432 heartbeater.cc:347] Connected to a master server at 127.15.113.190:34403
I20260809 22:57:36.730944 14432 heartbeater.cc:464] Registering TS with master...
I20260809 22:57:36.731810 14432 heartbeater.cc:511] Master 127.15.113.190:34403 requested a full tablet report, sending...
I20260809 22:57:36.733817 14131 ts_manager.cc:194] Registered new tserver with Master: 8c4d0483666a41b7acbf64634756da9e (127.15.113.129:34679)
I20260809 22:57:36.735404 14131 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.15.113.129:56433
W20260809 22:57:37.064327 14436 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:57:37.064594 14436 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:57:37.064721 14436 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:57:37.071602 14436 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:57:37.071692 14436 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:57:37.071832 14436 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.113.130
I20260809 22:57:37.083668 14436 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.113.148:38275
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786315935617919-15814-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786315935617919-15814-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.113.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786315935617919-15814-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.113.130
--webserver_port=0
--tserver_master_addrs=127.15.113.190:34403
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.113.130
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786315935617919-15814-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:57:37.085078 14436 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:57:37.086997 14436 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:57:37.099340 14444 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:57:37.099386 14449 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:57:37.099417 14445 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:57:37.100240 14436 server_base.cc:1034] running on GCE node
I20260809 22:57:37.100842 14436 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:57:37.101725 14436 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:57:37.102919 14436 hybrid_clock.cc:648] HybridClock initialized: now 1786316257102872 us; error 36 us; skew 500 ppm
I20260809 22:57:37.103241 14436 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 22:57:37.105361 14436 webserver.cc:460] Webserver started at http://127.15.113.130:34481/ using document root <none> and password file <none>
I20260809 22:57:37.105993 14436 fs_manager.cc:359] Metadata directory not provided
I20260809 22:57:37.106091 14436 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:57:37.106386 14436 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:57:37.109115 14436 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786315935617919-15814-0/minicluster-data/ts-1/data/instance:
uuid: "ee7325758af642a4b95e073121ddd1c5"
format_stamp: "Formatted at 2026-08-09 22:57:37 on dist-test-slave-ncp9"
server_key: "71934dd03dee66fe48634ee670db0841"
server_key_iv: "f446a9dec5af8edb4faca8b1e6f2ec52"
server_key_version: "encryptionkey@0"
I20260809 22:57:37.109853 14436 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786315935617919-15814-0/minicluster-data/ts-1/wal/instance:
uuid: "ee7325758af642a4b95e073121ddd1c5"
format_stamp: "Formatted at 2026-08-09 22:57:37 on dist-test-slave-ncp9"
server_key: "71934dd03dee66fe48634ee670db0841"
server_key_iv: "f446a9dec5af8edb4faca8b1e6f2ec52"
server_key_version: "encryptionkey@0"
I20260809 22:57:37.114756 14436 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 22:57:37.118005 14455 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:57:37.119242 14436 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 22:57:37.119386 14436 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786315935617919-15814-0/minicluster-data/ts-1/data,/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786315935617919-15814-0/minicluster-data/ts-1/wal
uuid: "ee7325758af642a4b95e073121ddd1c5"
format_stamp: "Formatted at 2026-08-09 22:57:37 on dist-test-slave-ncp9"
server_key: "71934dd03dee66fe48634ee670db0841"
server_key_iv: "f446a9dec5af8edb4faca8b1e6f2ec52"
server_key_version: "encryptionkey@0"
I20260809 22:57:37.119591 14436 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786315935617919-15814-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786315935617919-15814-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786315935617919-15814-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:57:37.133302 14436 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:57:37.134030 14436 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:57:37.134289 14436 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:57:37.135176 14436 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:57:37.136533 14436 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:57:37.136626 14436 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:57:37.136734 14436 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:57:37.136811 14436 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:57:37.169454 14436 rpc_server.cc:307] RPC server started. Bound to: 127.15.113.130:33169
I20260809 22:57:37.169512 14620 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.113.130:33169 every 8 connection(s)
I20260809 22:57:37.170778 14436 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786315935617919-15814-0/minicluster-data/ts-1/data/info.pb
I20260809 22:57:37.179831 15814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskItYPJR/build/asan/bin/kudu as pid 14436
I20260809 22:57:37.180003 15814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786315935617919-15814-0/minicluster-data/ts-1/wal/instance
I20260809 22:57:37.180426 15814 external_mini_cluster.cc:1428] Setting key 5bb967fa17c44cd4624964cc5af1226b
I20260809 22:57:37.182924 14434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:57:37.173299 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.113.130:33461 (local address 127.15.113.190:34403)
0809 22:57:37.173452 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0809 22:57:37.173456 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 22:57:37.175066 (+ 1610us) server_negotiation.cc:407] Connection header received
0809 22:57:37.176769 (+ 1703us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:57:37.176775 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:57:37.176853 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:57:37.176972 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:57:37.177969 (+ 997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:57:37.178695 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:57:37.180257 (+ 1562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:57:37.180670 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:57:37.180778 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:57:37.182225 (+ 1447us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:57:37.182231 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:57:37.182236 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 22:57:37.182257 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:57:37.182295 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 22:57:37.182313 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:57:37.182411 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0809 22:57:37.182634 (+ 223us) server_negotiation.cc:299] Negotiation successful
0809 22:57:37.182708 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260809 22:57:37.183611 15814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskItYPJR/build/asan/bin/kudu
/tmp/dist-test-taskItYPJR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786315935617919-15814-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786315935617919-15814-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786315935617919-15814-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786315935617919-15814-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.113.131:0
--local_ip_for_outbound_sockets=127.15.113.131
--webserver_interface=127.15.113.131
--webserver_port=0
--tserver_master_addrs=127.15.113.190:34403
--builtin_ntp_servers=127.15.113.148:38275
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260809 22:57:37.184187 14627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:57:37.173977 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:34403 (local address 127.15.113.130:33461)
0809 22:57:37.174903 (+ 926us) negotiation.cc:107] Waiting for socket to connect
0809 22:57:37.174949 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0809 22:57:37.176521 (+ 1572us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:57:37.177199 (+ 678us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:57:37.177207 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:57:37.177288 (+ 81us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:57:37.177809 (+ 521us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:57:37.177825 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:57:37.178847 (+ 1022us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:57:37.178851 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:57:37.180118 (+ 1267us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:57:37.180131 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:57:37.180884 (+ 753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:57:37.180914 (+ 30us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:57:37.181141 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:57:37.181872 (+ 731us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 22:57:37.181875 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:57:37.181912 (+ 37us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 22:57:37.182002 (+ 90us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:57:37.182418 (+ 416us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:57:37.182429 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:57:37.182437 (+ 8us) client_negotiation.cc:769] Sending connection context
0809 22:57:37.182623 (+ 186us) client_negotiation.cc:240] Negotiation successful
0809 22:57:37.182984 (+ 361us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":504,"thread_start_us":306,"threads_started":1}
I20260809 22:57:37.185479 14623 heartbeater.cc:347] Connected to a master server at 127.15.113.190:34403
I20260809 22:57:37.185770 14623 heartbeater.cc:464] Registering TS with master...
I20260809 22:57:37.186587 14623 heartbeater.cc:511] Master 127.15.113.190:34403 requested a full tablet report, sending...
I20260809 22:57:37.187917 14131 ts_manager.cc:194] Registered new tserver with Master: ee7325758af642a4b95e073121ddd1c5 (127.15.113.130:33169)
I20260809 22:57:37.188714 14131 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.15.113.130:33461
W20260809 22:57:37.514678 14628 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:57:37.514950 14628 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:57:37.515039 14628 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:57:37.522099 14628 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:57:37.522210 14628 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:57:37.522358 14628 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.113.131
I20260809 22:57:37.531026 14628 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.113.148:38275
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786315935617919-15814-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786315935617919-15814-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.113.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786315935617919-15814-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.113.131
--webserver_port=0
--tserver_master_addrs=127.15.113.190:34403
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.113.131
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786315935617919-15814-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:57:37.532191 14628 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:57:37.534085 14628 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:57:37.546655 14639 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:57:37.546730 14636 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:57:37.546710 14637 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:57:37.547554 14628 server_base.cc:1034] running on GCE node
I20260809 22:57:37.548137 14628 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:57:37.549024 14628 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:57:37.550223 14628 hybrid_clock.cc:648] HybridClock initialized: now 1786316257550174 us; error 41 us; skew 500 ppm
I20260809 22:57:37.550515 14628 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 22:57:37.552655 14628 webserver.cc:460] Webserver started at http://127.15.113.131:38487/ using document root <none> and password file <none>
I20260809 22:57:37.553349 14628 fs_manager.cc:359] Metadata directory not provided
I20260809 22:57:37.553455 14628 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:57:37.553768 14628 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:57:37.556667 14628 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786315935617919-15814-0/minicluster-data/ts-2/data/instance:
uuid: "adfc17940544486787ee0ede84ab1a3f"
format_stamp: "Formatted at 2026-08-09 22:57:37 on dist-test-slave-ncp9"
server_key: "0853bf4f8d7e9ead39cf4ddf082f5385"
server_key_iv: "a245a438cb326c378ba62068eca38df9"
server_key_version: "encryptionkey@0"
I20260809 22:57:37.557475 14628 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786315935617919-15814-0/minicluster-data/ts-2/wal/instance:
uuid: "adfc17940544486787ee0ede84ab1a3f"
format_stamp: "Formatted at 2026-08-09 22:57:37 on dist-test-slave-ncp9"
server_key: "0853bf4f8d7e9ead39cf4ddf082f5385"
server_key_iv: "a245a438cb326c378ba62068eca38df9"
server_key_version: "encryptionkey@0"
I20260809 22:57:37.562286 14628 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.003s
I20260809 22:57:37.565572 14648 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:57:37.566782 14628 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.004s sys 0.000s
I20260809 22:57:37.566922 14628 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786315935617919-15814-0/minicluster-data/ts-2/data,/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786315935617919-15814-0/minicluster-data/ts-2/wal
uuid: "adfc17940544486787ee0ede84ab1a3f"
format_stamp: "Formatted at 2026-08-09 22:57:37 on dist-test-slave-ncp9"
server_key: "0853bf4f8d7e9ead39cf4ddf082f5385"
server_key_iv: "a245a438cb326c378ba62068eca38df9"
server_key_version: "encryptionkey@0"
I20260809 22:57:37.567106 14628 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786315935617919-15814-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786315935617919-15814-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786315935617919-15814-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:57:37.584673 14628 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:57:37.585438 14628 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:57:37.585714 14628 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:57:37.586593 14628 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:57:37.587926 14628 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:57:37.588027 14628 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:57:37.588150 14628 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:57:37.588223 14628 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:57:37.618091 14628 rpc_server.cc:307] RPC server started. Bound to: 127.15.113.131:42979
I20260809 22:57:37.618140 14807 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.113.131:42979 every 8 connection(s)
I20260809 22:57:37.619364 14628 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786315935617919-15814-0/minicluster-data/ts-2/data/info.pb
I20260809 22:57:37.623832 15814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskItYPJR/build/asan/bin/kudu as pid 14628
I20260809 22:57:37.623996 15814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786315935617919-15814-0/minicluster-data/ts-2/wal/instance
I20260809 22:57:37.624435 15814 external_mini_cluster.cc:1428] Setting key 22799565a754b48713e567f5220579af
I20260809 22:57:37.631410 14434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:57:37.621862 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.113.131:36789 (local address 127.15.113.190:34403)
0809 22:57:37.622038 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0809 22:57:37.622043 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:57:37.623511 (+ 1468us) server_negotiation.cc:407] Connection header received
0809 22:57:37.625407 (+ 1896us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:57:37.625413 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:57:37.625497 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:57:37.625582 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:57:37.626628 (+ 1046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:57:37.627378 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:57:37.628877 (+ 1499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:57:37.629363 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:57:37.629472 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:57:37.630665 (+ 1193us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:57:37.630670 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:57:37.630673 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 22:57:37.630689 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:57:37.630720 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 22:57:37.630732 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:57:37.630802 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0809 22:57:37.631011 (+ 209us) server_negotiation.cc:299] Negotiation successful
0809 22:57:37.631104 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260809 22:57:37.632368 14810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:57:37.622425 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:34403 (local address 127.15.113.131:36789)
0809 22:57:37.623329 (+ 904us) negotiation.cc:107] Waiting for socket to connect
0809 22:57:37.623386 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0809 22:57:37.625119 (+ 1733us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:57:37.625824 (+ 705us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:57:37.625832 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:57:37.625919 (+ 87us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:57:37.626474 (+ 555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:57:37.626490 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:57:37.627523 (+ 1033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:57:37.627527 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:57:37.628742 (+ 1215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:57:37.628755 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:57:37.629572 (+ 817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:57:37.629579 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:57:37.629795 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:57:37.630365 (+ 570us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 22:57:37.630368 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:57:37.630403 (+ 35us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 22:57:37.630480 (+ 77us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:57:37.630818 (+ 338us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:57:37.630830 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:57:37.630837 (+ 7us) client_negotiation.cc:769] Sending connection context
0809 22:57:37.630982 (+ 145us) client_negotiation.cc:240] Negotiation successful
0809 22:57:37.631363 (+ 381us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":466,"thread_start_us":256,"threads_started":1}
I20260809 22:57:37.633569 14808 heartbeater.cc:347] Connected to a master server at 127.15.113.190:34403
I20260809 22:57:37.633808 14808 heartbeater.cc:464] Registering TS with master...
I20260809 22:57:37.634505 14808 heartbeater.cc:511] Master 127.15.113.190:34403 requested a full tablet report, sending...
I20260809 22:57:37.635807 14131 ts_manager.cc:194] Registered new tserver with Master: adfc17940544486787ee0ede84ab1a3f (127.15.113.131:42979)
I20260809 22:57:37.636577 14131 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.15.113.131:36789
I20260809 22:57:37.639072 15814 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 22:57:37.647984 14434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:57:37.642962 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35382 (local address 127.15.113.190:34403)
0809 22:57:37.643146 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0809 22:57:37.643160 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0809 22:57:37.643448 (+ 288us) server_negotiation.cc:407] Connection header received
0809 22:57:37.643575 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:57:37.643578 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:57:37.643646 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:57:37.643723 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:57:37.644113 (+ 390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:57:37.644770 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:57:37.646103 (+ 1333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:57:37.646510 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:57:37.646602 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:57:37.647470 (+ 868us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:57:37.647474 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:57:37.647477 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 22:57:37.647492 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:57:37.647516 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 22:57:37.647526 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:57:37.647583 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0809 22:57:37.647673 (+ 90us) server_negotiation.cc:299] Negotiation successful
0809 22:57:37.647733 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260809 22:57:37.651412 14131 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35382:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 22:57:37.658037 14131 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 22:57:37.680069 14829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:57:37.669379 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41016 (local address 127.15.113.131:42979)
0809 22:57:37.671438 (+ 2059us) server_negotiation.cc:206] Beginning negotiation
0809 22:57:37.671446 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 22:57:37.671480 (+ 34us) server_negotiation.cc:407] Connection header received
0809 22:57:37.671575 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:57:37.671582 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:57:37.671839 (+ 257us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:57:37.672793 (+ 954us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:57:37.673085 (+ 292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:57:37.674704 (+ 1619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:57:37.677788 (+ 3084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:57:37.678959 (+ 1171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:57:37.679138 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:57:37.679349 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0809 22:57:37.679552 (+ 203us) server_negotiation.cc:299] Negotiation successful
0809 22:57:37.679720 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1902,"thread_start_us":1746,"threads_started":1}
I20260809 22:57:37.680367 14828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:57:37.669153 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.131:42979 (local address 127.0.0.1:41016)
0809 22:57:37.670125 (+ 972us) negotiation.cc:107] Waiting for socket to connect
0809 22:57:37.670166 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0809 22:57:37.670345 (+ 179us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:57:37.672092 (+ 1747us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:57:37.672102 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:57:37.672141 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:57:37.672561 (+ 420us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:57:37.672575 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:57:37.674872 (+ 2297us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:57:37.674878 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:57:37.677543 (+ 2665us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:57:37.677558 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:57:37.679138 (+ 1580us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:57:37.679145 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:57:37.679335 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:57:37.679384 (+ 49us) client_negotiation.cc:769] Sending connection context
0809 22:57:37.679497 (+ 113us) client_negotiation.cc:240] Negotiation successful
0809 22:57:37.679677 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":721,"mutex_wait_us":84,"thread_start_us":2182,"threads_started":1}
I20260809 22:57:37.681259 14831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:57:37.670116 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46076 (local address 127.15.113.130:33169)
0809 22:57:37.670950 (+ 834us) server_negotiation.cc:206] Beginning negotiation
0809 22:57:37.670959 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 22:57:37.671695 (+ 736us) server_negotiation.cc:407] Connection header received
0809 22:57:37.672008 (+ 313us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:57:37.672015 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:57:37.672249 (+ 234us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:57:37.672534 (+ 285us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:57:37.673503 (+ 969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:57:37.675182 (+ 1679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:57:37.678432 (+ 3250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:57:37.679751 (+ 1319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:57:37.679919 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:57:37.680103 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0809 22:57:37.680691 (+ 588us) server_negotiation.cc:299] Negotiation successful
0809 22:57:37.680846 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":646,"thread_start_us":917,"threads_started":1}
I20260809 22:57:37.681926 14827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:57:37.669406 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.130:33169 (local address 127.0.0.1:46076)
0809 22:57:37.671581 (+ 2175us) negotiation.cc:107] Waiting for socket to connect
0809 22:57:37.671595 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 22:57:37.671810 (+ 215us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:57:37.672873 (+ 1063us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:57:37.672878 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:57:37.672925 (+ 47us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:57:37.673320 (+ 395us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:57:37.673331 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:57:37.675382 (+ 2051us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:57:37.675387 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:57:37.678246 (+ 2859us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:57:37.678259 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:57:37.680259 (+ 2000us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:57:37.680265 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:57:37.680468 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:57:37.680504 (+ 36us) client_negotiation.cc:769] Sending connection context
0809 22:57:37.681442 (+ 938us) client_negotiation.cc:240] Negotiation successful
0809 22:57:37.681571 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2064,"mutex_wait_us":523,"thread_start_us":956,"threads_started":1}
I20260809 22:57:37.684924 14836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:57:37.671937 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58714 (local address 127.15.113.129:34679)
0809 22:57:37.676816 (+ 4879us) server_negotiation.cc:206] Beginning negotiation
0809 22:57:37.676824 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 22:57:37.677449 (+ 625us) server_negotiation.cc:407] Connection header received
0809 22:57:37.677550 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:57:37.677557 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:57:37.677747 (+ 190us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:57:37.677927 (+ 180us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:57:37.678608 (+ 681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:57:37.680035 (+ 1427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:57:37.682501 (+ 2466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:57:37.683552 (+ 1051us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:57:37.683711 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:57:37.683900 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0809 22:57:37.684111 (+ 211us) server_negotiation.cc:299] Negotiation successful
0809 22:57:37.684270 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4739,"thread_start_us":4640,"threads_started":1}
I20260809 22:57:37.684988 14832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:57:37.671702 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.129:34679 (local address 127.0.0.1:58714)
0809 22:57:37.677197 (+ 5495us) negotiation.cc:107] Waiting for socket to connect
0809 22:57:37.677215 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 22:57:37.677332 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:57:37.677927 (+ 595us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:57:37.677931 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:57:37.677958 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:57:37.678433 (+ 475us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:57:37.678446 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:57:37.680189 (+ 1743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:57:37.680194 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:57:37.682337 (+ 2143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:57:37.682350 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:57:37.683750 (+ 1400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:57:37.683756 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:57:37.683921 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:57:37.683953 (+ 32us) client_negotiation.cc:769] Sending connection context
0809 22:57:37.684049 (+ 96us) client_negotiation.cc:240] Negotiation successful
0809 22:57:37.684171 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5361,"mutex_wait_us":3,"thread_start_us":5220,"threads_started":1}
I20260809 22:57:37.684516 14713 tablet_service.cc:1467] Processing CreateTablet for tablet c93200a7ad864f548ae1e3faa23b0dae (DEFAULT_TABLE table=test-table [id=502ee32856864ccf94391a53fb036337]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:57:37.686017 14713 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c93200a7ad864f548ae1e3faa23b0dae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:57:37.685847 14527 tablet_service.cc:1467] Processing CreateTablet for tablet c93200a7ad864f548ae1e3faa23b0dae (DEFAULT_TABLE table=test-table [id=502ee32856864ccf94391a53fb036337]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:57:37.686976 14527 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c93200a7ad864f548ae1e3faa23b0dae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:57:37.688537 14330 tablet_service.cc:1467] Processing CreateTablet for tablet c93200a7ad864f548ae1e3faa23b0dae (DEFAULT_TABLE table=test-table [id=502ee32856864ccf94391a53fb036337]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:57:37.689879 14330 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c93200a7ad864f548ae1e3faa23b0dae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:57:37.697762 14837 tablet_bootstrap.cc:492] T c93200a7ad864f548ae1e3faa23b0dae P ee7325758af642a4b95e073121ddd1c5: Bootstrap starting.
I20260809 22:57:37.701538 14837 tablet_bootstrap.cc:654] T c93200a7ad864f548ae1e3faa23b0dae P ee7325758af642a4b95e073121ddd1c5: Neither blocks nor log segments found. Creating new log.
I20260809 22:57:37.701526 14839 tablet_bootstrap.cc:492] T c93200a7ad864f548ae1e3faa23b0dae P adfc17940544486787ee0ede84ab1a3f: Bootstrap starting.
I20260809 22:57:37.702973 14837 log.cc:824] T c93200a7ad864f548ae1e3faa23b0dae P ee7325758af642a4b95e073121ddd1c5: Log is configured to *not* fsync() on all Append() calls
I20260809 22:57:37.704872 14840 tablet_bootstrap.cc:492] T c93200a7ad864f548ae1e3faa23b0dae P 8c4d0483666a41b7acbf64634756da9e: Bootstrap starting.
I20260809 22:57:37.705633 14839 tablet_bootstrap.cc:654] T c93200a7ad864f548ae1e3faa23b0dae P adfc17940544486787ee0ede84ab1a3f: Neither blocks nor log segments found. Creating new log.
I20260809 22:57:37.706915 14839 log.cc:824] T c93200a7ad864f548ae1e3faa23b0dae P adfc17940544486787ee0ede84ab1a3f: Log is configured to *not* fsync() on all Append() calls
I20260809 22:57:37.707824 14837 tablet_bootstrap.cc:492] T c93200a7ad864f548ae1e3faa23b0dae P ee7325758af642a4b95e073121ddd1c5: No bootstrap required, opened a new log
I20260809 22:57:37.707917 14840 tablet_bootstrap.cc:654] T c93200a7ad864f548ae1e3faa23b0dae P 8c4d0483666a41b7acbf64634756da9e: Neither blocks nor log segments found. Creating new log.
I20260809 22:57:37.708092 14837 ts_tablet_manager.cc:1397] T c93200a7ad864f548ae1e3faa23b0dae P ee7325758af642a4b95e073121ddd1c5: Time spent bootstrapping tablet: real 0.011s user 0.009s sys 0.000s
I20260809 22:57:37.708928 14840 log.cc:824] T c93200a7ad864f548ae1e3faa23b0dae P 8c4d0483666a41b7acbf64634756da9e: Log is configured to *not* fsync() on all Append() calls
I20260809 22:57:37.709983 14839 tablet_bootstrap.cc:492] T c93200a7ad864f548ae1e3faa23b0dae P adfc17940544486787ee0ede84ab1a3f: No bootstrap required, opened a new log
I20260809 22:57:37.710246 14839 ts_tablet_manager.cc:1397] T c93200a7ad864f548ae1e3faa23b0dae P adfc17940544486787ee0ede84ab1a3f: Time spent bootstrapping tablet: real 0.009s user 0.004s sys 0.003s
I20260809 22:57:37.711552 14840 tablet_bootstrap.cc:492] T c93200a7ad864f548ae1e3faa23b0dae P 8c4d0483666a41b7acbf64634756da9e: No bootstrap required, opened a new log
I20260809 22:57:37.711784 14840 ts_tablet_manager.cc:1397] T c93200a7ad864f548ae1e3faa23b0dae P 8c4d0483666a41b7acbf64634756da9e: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.003s
I20260809 22:57:37.718295 14837 raft_consensus.cc:348] T c93200a7ad864f548ae1e3faa23b0dae P ee7325758af642a4b95e073121ddd1c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "adfc17940544486787ee0ede84ab1a3f" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 42979 } } peers { permanent_uuid: "ee7325758af642a4b95e073121ddd1c5" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 33169 } } peers { permanent_uuid: "8c4d0483666a41b7acbf64634756da9e" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 34679 } }
I20260809 22:57:37.718741 14837 raft_consensus.cc:374] T c93200a7ad864f548ae1e3faa23b0dae P ee7325758af642a4b95e073121ddd1c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:57:37.718827 14837 raft_consensus.cc:729] T c93200a7ad864f548ae1e3faa23b0dae P ee7325758af642a4b95e073121ddd1c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee7325758af642a4b95e073121ddd1c5, State: Initialized, Role: FOLLOWER
I20260809 22:57:37.719436 14837 consensus_queue.cc:260] T c93200a7ad864f548ae1e3faa23b0dae P ee7325758af642a4b95e073121ddd1c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "adfc17940544486787ee0ede84ab1a3f" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 42979 } } peers { permanent_uuid: "ee7325758af642a4b95e073121ddd1c5" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 33169 } } peers { permanent_uuid: "8c4d0483666a41b7acbf64634756da9e" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 34679 } }
I20260809 22:57:37.720435 14839 raft_consensus.cc:348] T c93200a7ad864f548ae1e3faa23b0dae P adfc17940544486787ee0ede84ab1a3f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "adfc17940544486787ee0ede84ab1a3f" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 42979 } } peers { permanent_uuid: "ee7325758af642a4b95e073121ddd1c5" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 33169 } } peers { permanent_uuid: "8c4d0483666a41b7acbf64634756da9e" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 34679 } }
I20260809 22:57:37.720863 14839 raft_consensus.cc:374] T c93200a7ad864f548ae1e3faa23b0dae P adfc17940544486787ee0ede84ab1a3f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:57:37.721005 14839 raft_consensus.cc:729] T c93200a7ad864f548ae1e3faa23b0dae P adfc17940544486787ee0ede84ab1a3f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: adfc17940544486787ee0ede84ab1a3f, State: Initialized, Role: FOLLOWER
I20260809 22:57:37.721692 14839 consensus_queue.cc:260] T c93200a7ad864f548ae1e3faa23b0dae P adfc17940544486787ee0ede84ab1a3f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "adfc17940544486787ee0ede84ab1a3f" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 42979 } } peers { permanent_uuid: "ee7325758af642a4b95e073121ddd1c5" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 33169 } } peers { permanent_uuid: "8c4d0483666a41b7acbf64634756da9e" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 34679 } }
I20260809 22:57:37.724056 14840 raft_consensus.cc:348] T c93200a7ad864f548ae1e3faa23b0dae P 8c4d0483666a41b7acbf64634756da9e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "adfc17940544486787ee0ede84ab1a3f" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 42979 } } peers { permanent_uuid: "ee7325758af642a4b95e073121ddd1c5" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 33169 } } peers { permanent_uuid: "8c4d0483666a41b7acbf64634756da9e" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 34679 } }
I20260809 22:57:37.724540 14840 raft_consensus.cc:374] T c93200a7ad864f548ae1e3faa23b0dae P 8c4d0483666a41b7acbf64634756da9e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:57:37.724643 14840 raft_consensus.cc:729] T c93200a7ad864f548ae1e3faa23b0dae P 8c4d0483666a41b7acbf64634756da9e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c4d0483666a41b7acbf64634756da9e, State: Initialized, Role: FOLLOWER
I20260809 22:57:37.725284 14840 consensus_queue.cc:260] T c93200a7ad864f548ae1e3faa23b0dae P 8c4d0483666a41b7acbf64634756da9e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "adfc17940544486787ee0ede84ab1a3f" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 42979 } } peers { permanent_uuid: "ee7325758af642a4b95e073121ddd1c5" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 33169 } } peers { permanent_uuid: "8c4d0483666a41b7acbf64634756da9e" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 34679 } }
I20260809 22:57:37.726012 14808 heartbeater.cc:503] Master 127.15.113.190:34403 was elected leader, sending a full tablet report...
I20260809 22:57:37.727020 14839 ts_tablet_manager.cc:1428] T c93200a7ad864f548ae1e3faa23b0dae P adfc17940544486787ee0ede84ab1a3f: Time spent starting tablet: real 0.017s user 0.016s sys 0.000s
I20260809 22:57:37.727823 14623 heartbeater.cc:503] Master 127.15.113.190:34403 was elected leader, sending a full tablet report...
I20260809 22:57:37.729660 14837 ts_tablet_manager.cc:1428] T c93200a7ad864f548ae1e3faa23b0dae P ee7325758af642a4b95e073121ddd1c5: Time spent starting tablet: real 0.021s user 0.019s sys 0.000s
I20260809 22:57:37.730520 14432 heartbeater.cc:503] Master 127.15.113.190:34403 was elected leader, sending a full tablet report...
I20260809 22:57:37.732153 14840 ts_tablet_manager.cc:1428] T c93200a7ad864f548ae1e3faa23b0dae P 8c4d0483666a41b7acbf64634756da9e: Time spent starting tablet: real 0.020s user 0.010s sys 0.005s
I20260809 22:57:37.771641 14848 raft_consensus.cc:482] T c93200a7ad864f548ae1e3faa23b0dae P 8c4d0483666a41b7acbf64634756da9e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 22:57:37.771941 14848 raft_consensus.cc:504] T c93200a7ad864f548ae1e3faa23b0dae P 8c4d0483666a41b7acbf64634756da9e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "adfc17940544486787ee0ede84ab1a3f" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 42979 } } peers { permanent_uuid: "ee7325758af642a4b95e073121ddd1c5" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 33169 } } peers { permanent_uuid: "8c4d0483666a41b7acbf64634756da9e" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 34679 } }
I20260809 22:57:37.773586 14848 leader_election.cc:288] T c93200a7ad864f548ae1e3faa23b0dae P 8c4d0483666a41b7acbf64634756da9e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers adfc17940544486787ee0ede84ab1a3f (127.15.113.131:42979), ee7325758af642a4b95e073121ddd1c5 (127.15.113.130:33169)
I20260809 22:57:37.782016 14831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:57:37.774108 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.113.129:47463 (local address 127.15.113.130:33169)
0809 22:57:37.774279 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0809 22:57:37.774285 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:57:37.774927 (+ 642us) server_negotiation.cc:407] Connection header received
0809 22:57:37.775132 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:57:37.775136 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:57:37.775206 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:57:37.775289 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:57:37.775958 (+ 669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:57:37.777064 (+ 1106us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:57:37.779861 (+ 2797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:57:37.781068 (+ 1207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:57:37.781228 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:57:37.781297 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0809 22:57:37.781644 (+ 347us) server_negotiation.cc:299] Negotiation successful
0809 22:57:37.781771 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260809 22:57:37.782009 14829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:57:37.773877 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.113.129:35399 (local address 127.15.113.131:42979)
0809 22:57:37.774067 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0809 22:57:37.774074 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:57:37.774940 (+ 866us) server_negotiation.cc:407] Connection header received
0809 22:57:37.775095 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:57:37.775100 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:57:37.775188 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:57:37.775288 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:57:37.776005 (+ 717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:57:37.777064 (+ 1059us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:57:37.779864 (+ 2800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:57:37.781068 (+ 1204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:57:37.781228 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:57:37.781297 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0809 22:57:37.781650 (+ 353us) server_negotiation.cc:299] Negotiation successful
0809 22:57:37.781771 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260809 22:57:37.782202 14849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:57:37.774071 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.131:42979 (local address 127.15.113.129:35399)
0809 22:57:37.774845 (+ 774us) negotiation.cc:107] Waiting for socket to connect
0809 22:57:37.774864 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 22:57:37.774971 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:57:37.775302 (+ 331us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:57:37.775307 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:57:37.775347 (+ 40us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:57:37.775850 (+ 503us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:57:37.775860 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:57:37.777183 (+ 1323us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:57:37.777188 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:57:37.779658 (+ 2470us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:57:37.779673 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:57:37.781242 (+ 1569us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:57:37.781249 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:57:37.781423 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:57:37.781512 (+ 89us) client_negotiation.cc:769] Sending connection context
0809 22:57:37.781596 (+ 84us) client_negotiation.cc:240] Negotiation successful
0809 22:57:37.781739 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":675,"mutex_wait_us":188,"spinlock_wait_cycles":3456,"thread_start_us":364,"threads_started":1}
I20260809 22:57:37.782492 14850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:57:37.774013 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.130:33169 (local address 127.15.113.129:47463)
0809 22:57:37.774804 (+ 791us) negotiation.cc:107] Waiting for socket to connect
0809 22:57:37.774823 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 22:57:37.775012 (+ 189us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:57:37.775302 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:57:37.775307 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:57:37.775338 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:57:37.775820 (+ 482us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:57:37.775837 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:57:37.777186 (+ 1349us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:57:37.777190 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:57:37.779660 (+ 2470us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:57:37.779673 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:57:37.781280 (+ 1607us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:57:37.781287 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:57:37.781438 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:57:37.781509 (+ 71us) client_negotiation.cc:769] Sending connection context
0809 22:57:37.781600 (+ 91us) client_negotiation.cc:240] Negotiation successful
0809 22:57:37.781739 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":650,"mutex_wait_us":181,"spinlock_wait_cycles":34048,"thread_start_us":431,"threads_started":1}
I20260809 22:57:37.783186 14743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c93200a7ad864f548ae1e3faa23b0dae" candidate_uuid: "8c4d0483666a41b7acbf64634756da9e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "adfc17940544486787ee0ede84ab1a3f" is_pre_election: true
I20260809 22:57:37.783394 14560 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c93200a7ad864f548ae1e3faa23b0dae" candidate_uuid: "8c4d0483666a41b7acbf64634756da9e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee7325758af642a4b95e073121ddd1c5" is_pre_election: true
I20260809 22:57:37.783654 14743 raft_consensus.cc:2454] T c93200a7ad864f548ae1e3faa23b0dae P adfc17940544486787ee0ede84ab1a3f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c4d0483666a41b7acbf64634756da9e in term 0.
I20260809 22:57:37.783908 14560 raft_consensus.cc:2454] T c93200a7ad864f548ae1e3faa23b0dae P ee7325758af642a4b95e073121ddd1c5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c4d0483666a41b7acbf64634756da9e in term 0.
I20260809 22:57:37.784399 14256 leader_election.cc:302] T c93200a7ad864f548ae1e3faa23b0dae P 8c4d0483666a41b7acbf64634756da9e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c4d0483666a41b7acbf64634756da9e, adfc17940544486787ee0ede84ab1a3f; no voters:
I20260809 22:57:37.784833 14848 raft_consensus.cc:2781] T c93200a7ad864f548ae1e3faa23b0dae P 8c4d0483666a41b7acbf64634756da9e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 22:57:37.785006 14848 raft_consensus.cc:482] T c93200a7ad864f548ae1e3faa23b0dae P 8c4d0483666a41b7acbf64634756da9e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 22:57:37.785106 14848 raft_consensus.cc:3037] T c93200a7ad864f548ae1e3faa23b0dae P 8c4d0483666a41b7acbf64634756da9e [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:57:37.787176 14848 raft_consensus.cc:504] T c93200a7ad864f548ae1e3faa23b0dae P 8c4d0483666a41b7acbf64634756da9e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "adfc17940544486787ee0ede84ab1a3f" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 42979 } } peers { permanent_uuid: "ee7325758af642a4b95e073121ddd1c5" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 33169 } } peers { permanent_uuid: "8c4d0483666a41b7acbf64634756da9e" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 34679 } }
I20260809 22:57:37.787740 14848 leader_election.cc:288] T c93200a7ad864f548ae1e3faa23b0dae P 8c4d0483666a41b7acbf64634756da9e [CANDIDATE]: Term 1 election: Requested vote from peers adfc17940544486787ee0ede84ab1a3f (127.15.113.131:42979), ee7325758af642a4b95e073121ddd1c5 (127.15.113.130:33169)
I20260809 22:57:37.788139 14743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c93200a7ad864f548ae1e3faa23b0dae" candidate_uuid: "8c4d0483666a41b7acbf64634756da9e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "adfc17940544486787ee0ede84ab1a3f"
I20260809 22:57:37.788192 14560 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c93200a7ad864f548ae1e3faa23b0dae" candidate_uuid: "8c4d0483666a41b7acbf64634756da9e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee7325758af642a4b95e073121ddd1c5"
I20260809 22:57:37.788355 14743 raft_consensus.cc:3037] T c93200a7ad864f548ae1e3faa23b0dae P adfc17940544486787ee0ede84ab1a3f [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:57:37.788355 14560 raft_consensus.cc:3037] T c93200a7ad864f548ae1e3faa23b0dae P ee7325758af642a4b95e073121ddd1c5 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:57:37.791528 14560 raft_consensus.cc:2454] T c93200a7ad864f548ae1e3faa23b0dae P ee7325758af642a4b95e073121ddd1c5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c4d0483666a41b7acbf64634756da9e in term 1.
I20260809 22:57:37.791528 14743 raft_consensus.cc:2454] T c93200a7ad864f548ae1e3faa23b0dae P adfc17940544486787ee0ede84ab1a3f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c4d0483666a41b7acbf64634756da9e in term 1.
I20260809 22:57:37.791913 14256 leader_election.cc:302] T c93200a7ad864f548ae1e3faa23b0dae P 8c4d0483666a41b7acbf64634756da9e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 8c4d0483666a41b7acbf64634756da9e, adfc17940544486787ee0ede84ab1a3f, ee7325758af642a4b95e073121ddd1c5; no voters:
I20260809 22:57:37.792201 14848 raft_consensus.cc:2781] T c93200a7ad864f548ae1e3faa23b0dae P 8c4d0483666a41b7acbf64634756da9e [term 1 FOLLOWER]: Leader election won for term 1
I20260809 22:57:37.792608 14848 raft_consensus.cc:686] T c93200a7ad864f548ae1e3faa23b0dae P 8c4d0483666a41b7acbf64634756da9e [term 1 LEADER]: Becoming Leader. State: Replica: 8c4d0483666a41b7acbf64634756da9e, State: Running, Role: LEADER
I20260809 22:57:37.793092 14848 consensus_queue.cc:237] T c93200a7ad864f548ae1e3faa23b0dae P 8c4d0483666a41b7acbf64634756da9e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "adfc17940544486787ee0ede84ab1a3f" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 42979 } } peers { permanent_uuid: "ee7325758af642a4b95e073121ddd1c5" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 33169 } } peers { permanent_uuid: "8c4d0483666a41b7acbf64634756da9e" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 34679 } }
I20260809 22:57:37.797142 14131 catalog_manager.cc:5495] T c93200a7ad864f548ae1e3faa23b0dae P 8c4d0483666a41b7acbf64634756da9e reported cstate change: term changed from 0 to 1, leader changed from <none> to 8c4d0483666a41b7acbf64634756da9e (127.15.113.129). New cstate: current_term: 1 leader_uuid: "8c4d0483666a41b7acbf64634756da9e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "adfc17940544486787ee0ede84ab1a3f" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 42979 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ee7325758af642a4b95e073121ddd1c5" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 33169 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8c4d0483666a41b7acbf64634756da9e" member_type: VOTER last_known_addr { host: "127.15.113.129" port: 34679 } health_report { overall_health: HEALTHY } } }
I20260809 22:57:37.819016 14836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:57:37.814308 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58716 (local address 127.15.113.129:34679)
0809 22:57:37.814481 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0809 22:57:37.814486 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:57:37.814625 (+ 139us) server_negotiation.cc:407] Connection header received
0809 22:57:37.814823 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:57:37.814826 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:57:37.814907 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:57:37.815022 (+ 115us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 22:57:37.815300 (+ 278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:57:37.815986 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:57:37.817275 (+ 1289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:57:37.817678 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:57:37.817761 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:57:37.818138 (+ 377us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 22:57:37.818601 (+ 463us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 22:57:37.818715 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0809 22:57:37.818753 (+ 38us) server_negotiation.cc:299] Negotiation successful
0809 22:57:37.818822 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260809 22:57:37.827036 14560 raft_consensus.cc:1261] T c93200a7ad864f548ae1e3faa23b0dae P ee7325758af642a4b95e073121ddd1c5 [term 1 FOLLOWER]: Refusing update from remote peer 8c4d0483666a41b7acbf64634756da9e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 22:57:37.827039 14743 raft_consensus.cc:1261] T c93200a7ad864f548ae1e3faa23b0dae P adfc17940544486787ee0ede84ab1a3f [term 1 FOLLOWER]: Refusing update from remote peer 8c4d0483666a41b7acbf64634756da9e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 22:57:37.828070 14851 consensus_queue.cc:1035] T c93200a7ad864f548ae1e3faa23b0dae P 8c4d0483666a41b7acbf64634756da9e [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee7325758af642a4b95e073121ddd1c5" member_type: VOTER last_known_addr { host: "127.15.113.130" port: 33169 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 22:57:37.828399 14848 consensus_queue.cc:1035] T c93200a7ad864f548ae1e3faa23b0dae P 8c4d0483666a41b7acbf64634756da9e [LEADER]: Connected to new peer: Peer: permanent_uuid: "adfc17940544486787ee0ede84ab1a3f" member_type: VOTER last_known_addr { host: "127.15.113.131" port: 42979 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 22:57:37.839047 14855 mvcc.cc:205] Tried to move back new op lower bound from 7316751392048943104 to 7316751391923748864. Current Snapshot: MvccSnapshot[applied={T|T < 7316751392048943104}]
I20260809 22:57:37.840035 14857 mvcc.cc:205] Tried to move back new op lower bound from 7316751392048943104 to 7316751391923748864. Current Snapshot: MvccSnapshot[applied={T|T < 7316751392048943104}]
I20260809 22:57:37.855345 14124 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:35382:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 22:57:37.855594 14124 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:35382:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 22:57:37.859136 14124 catalog_manager.cc:5782] T 00000000000000000000000000000000 P eb4a5fe13a9145288382f79b64f52f2b: Sending DeleteTablet for 3 replicas of tablet c93200a7ad864f548ae1e3faa23b0dae
I20260809 22:57:37.860194 14527 tablet_service.cc:1514] Processing DeleteTablet for tablet c93200a7ad864f548ae1e3faa23b0dae with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:57:37 UTC) from {username='slave'} at 127.0.0.1:46076
I20260809 22:57:37.860275 14713 tablet_service.cc:1514] Processing DeleteTablet for tablet c93200a7ad864f548ae1e3faa23b0dae with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:57:37 UTC) from {username='slave'} at 127.0.0.1:41016
I20260809 22:57:37.860312 14330 tablet_service.cc:1514] Processing DeleteTablet for tablet c93200a7ad864f548ae1e3faa23b0dae with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:57:37 UTC) from {username='slave'} at 127.0.0.1:58714
I20260809 22:57:37.862320 15814 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskItYPJR/build/asan/bin/kudu pid 14232
I20260809 22:57:37.872725 14868 tablet_replica.cc:330] T c93200a7ad864f548ae1e3faa23b0dae P adfc17940544486787ee0ede84ab1a3f: stopping tablet replica
I20260809 22:57:37.873286 14868 raft_consensus.cc:2229] T c93200a7ad864f548ae1e3faa23b0dae P adfc17940544486787ee0ede84ab1a3f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 22:57:37.873871 14868 raft_consensus.cc:2258] T c93200a7ad864f548ae1e3faa23b0dae P adfc17940544486787ee0ede84ab1a3f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 22:57:37.874857 14869 tablet_replica.cc:330] T c93200a7ad864f548ae1e3faa23b0dae P 8c4d0483666a41b7acbf64634756da9e: stopping tablet replica
I20260809 22:57:37.875385 14867 tablet_replica.cc:330] T c93200a7ad864f548ae1e3faa23b0dae P ee7325758af642a4b95e073121ddd1c5: stopping tablet replica
I20260809 22:57:37.875825 14867 raft_consensus.cc:2229] T c93200a7ad864f548ae1e3faa23b0dae P ee7325758af642a4b95e073121ddd1c5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 22:57:37.876125 14868 ts_tablet_manager.cc:1905] T c93200a7ad864f548ae1e3faa23b0dae P adfc17940544486787ee0ede84ab1a3f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 22:57:37.876365 14869 raft_consensus.cc:2229] T c93200a7ad864f548ae1e3faa23b0dae P 8c4d0483666a41b7acbf64634756da9e [term 1 LEADER]: Raft consensus shutting down.
I20260809 22:57:37.876886 14867 raft_consensus.cc:2258] T c93200a7ad864f548ae1e3faa23b0dae P ee7325758af642a4b95e073121ddd1c5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 22:57:37.877581 14869 raft_consensus.cc:2258] T c93200a7ad864f548ae1e3faa23b0dae P 8c4d0483666a41b7acbf64634756da9e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 22:57:37.879783 14867 ts_tablet_manager.cc:1905] T c93200a7ad864f548ae1e3faa23b0dae P ee7325758af642a4b95e073121ddd1c5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 22:57:37.880203 14836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:57:37.864095 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58728 (local address 127.15.113.129:34679)
0809 22:57:37.864265 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0809 22:57:37.864277 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 22:57:37.868998 (+ 4721us) server_negotiation.cc:407] Connection header received
0809 22:57:37.869867 (+ 869us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:57:37.869874 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:57:37.869978 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:57:37.870052 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:57:37.872686 (+ 2634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:57:37.874064 (+ 1378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:57:37.875987 (+ 1923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:57:37.876630 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:57:37.876726 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:57:37.878561 (+ 1835us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:57:37.878572 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:57:37.878588 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 22:57:37.879300 (+ 712us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:57:37.879356 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 22:57:37.879381 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:57:37.879460 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0809 22:57:37.879713 (+ 253us) server_negotiation.cc:299] Negotiation successful
0809 22:57:37.879812 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260809 22:57:37.881358 14424 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:57:37.882400 14868 ts_tablet_manager.cc:1918] T c93200a7ad864f548ae1e3faa23b0dae P adfc17940544486787ee0ede84ab1a3f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 22:57:37.882543 14868 log.cc:1196] T c93200a7ad864f548ae1e3faa23b0dae P adfc17940544486787ee0ede84ab1a3f: Deleting WAL directory at /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786315935617919-15814-0/minicluster-data/ts-2/wal/wals/c93200a7ad864f548ae1e3faa23b0dae
I20260809 22:57:37.882963 14868 ts_tablet_manager.cc:1939] T c93200a7ad864f548ae1e3faa23b0dae P adfc17940544486787ee0ede84ab1a3f: Deleting consensus metadata
I20260809 22:57:37.884409 14099 catalog_manager.cc:4841] TS adfc17940544486787ee0ede84ab1a3f (127.15.113.131:42979): tablet c93200a7ad864f548ae1e3faa23b0dae (table test-table [id=502ee32856864ccf94391a53fb036337]) successfully deleted
I20260809 22:57:37.888435 14867 ts_tablet_manager.cc:1918] T c93200a7ad864f548ae1e3faa23b0dae P ee7325758af642a4b95e073121ddd1c5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 22:57:37.888564 14867 log.cc:1196] T c93200a7ad864f548ae1e3faa23b0dae P ee7325758af642a4b95e073121ddd1c5: Deleting WAL directory at /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786315935617919-15814-0/minicluster-data/ts-1/wal/wals/c93200a7ad864f548ae1e3faa23b0dae
I20260809 22:57:37.889002 14867 ts_tablet_manager.cc:1939] T c93200a7ad864f548ae1e3faa23b0dae P ee7325758af642a4b95e073121ddd1c5: Deleting consensus metadata
I20260809 22:57:37.890336 14100 catalog_manager.cc:4841] TS ee7325758af642a4b95e073121ddd1c5 (127.15.113.130:33169): tablet c93200a7ad864f548ae1e3faa23b0dae (table test-table [id=502ee32856864ccf94391a53fb036337]) successfully deleted
I20260809 22:57:37.885758 14869 ts_tablet_manager.cc:1905] T c93200a7ad864f548ae1e3faa23b0dae P 8c4d0483666a41b7acbf64634756da9e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 22:57:37.981067 15814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskItYPJR/build/asan/bin/kudu with pid 14232
W20260809 22:57:38.008245 14099 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.15.113.129:34679 (error 108)
I20260809 22:57:38.008486 15814 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskItYPJR/build/asan/bin/kudu pid 14436
I20260809 22:57:38.010180 14832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:57:38.009750 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.129:34679 (local address 127.0.0.1:58734)
0809 22:57:38.009912 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0809 22:57:38.010027 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.113.129:34679: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260809 22:57:38.010406 14099 catalog_manager.cc:4584] TS 8c4d0483666a41b7acbf64634756da9e (127.15.113.129:34679): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet c93200a7ad864f548ae1e3faa23b0dae: Network error: Client connection negotiation failed: client connection to 127.15.113.129:34679: connect: Connection refused (error 111)
I20260809 22:57:38.015328 14831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:57:38.009395 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46084 (local address 127.15.113.130:33169)
0809 22:57:38.009597 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0809 22:57:38.009603 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:57:38.009624 (+ 21us) server_negotiation.cc:407] Connection header received
0809 22:57:38.009728 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:57:38.009734 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:57:38.009819 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:57:38.009903 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:57:38.010384 (+ 481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:57:38.011189 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:57:38.012447 (+ 1258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:57:38.012845 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:57:38.013020 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:57:38.014027 (+ 1007us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:57:38.014035 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:57:38.014050 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 22:57:38.014648 (+ 598us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:57:38.014701 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 22:57:38.014721 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:57:38.014808 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0809 22:57:38.014890 (+ 82us) server_negotiation.cc:299] Negotiation successful
0809 22:57:38.014985 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260809 22:57:38.016027 14615 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:57:38.064057 14832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:57:38.063577 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.129:34679 (local address 127.0.0.1:58738)
0809 22:57:38.063797 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0809 22:57:38.063889 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.113.129:34679: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":84}
I20260809 22:57:38.065150 14832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:57:38.064750 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.129:34679 (local address 127.0.0.1:58742)
0809 22:57:38.064942 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0809 22:57:38.065018 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.113.129:34679: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260809 22:57:38.102912 15814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskItYPJR/build/asan/bin/kudu with pid 14436
I20260809 22:57:38.125782 14832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:57:38.125201 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.129:34679 (local address 127.0.0.1:58754)
0809 22:57:38.125456 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0809 22:57:38.125573 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.113.129:34679: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
I20260809 22:57:38.126832 14832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:57:38.126472 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.129:34679 (local address 127.0.0.1:58760)
0809 22:57:38.126616 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0809 22:57:38.126703 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.113.129:34679: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260809 22:57:38.128261 15814 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskItYPJR/build/asan/bin/kudu pid 14628
I20260809 22:57:38.134689 14829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:57:38.128955 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41032 (local address 127.15.113.131:42979)
0809 22:57:38.129155 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0809 22:57:38.129159 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 22:57:38.129179 (+ 20us) server_negotiation.cc:407] Connection header received
0809 22:57:38.129304 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:57:38.129307 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:57:38.129392 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:57:38.129473 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:57:38.129802 (+ 329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:57:38.130531 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:57:38.131754 (+ 1223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:57:38.132151 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:57:38.132242 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:57:38.133298 (+ 1056us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:57:38.133307 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:57:38.133324 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 22:57:38.133949 (+ 625us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:57:38.134005 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 22:57:38.134026 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:57:38.134132 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0809 22:57:38.134300 (+ 168us) server_negotiation.cc:299] Negotiation successful
0809 22:57:38.134390 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260809 22:57:38.135385 14803 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:57:38.207538 14832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:57:38.207010 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.129:34679 (local address 127.0.0.1:58764)
0809 22:57:38.207226 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0809 22:57:38.207387 (+ 161us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.113.129:34679: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260809 22:57:38.208532 14832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:57:38.208157 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.129:34679 (local address 127.0.0.1:58772)
0809 22:57:38.208350 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0809 22:57:38.208424 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.113.129:34679: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":90}
I20260809 22:57:38.221719 15814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskItYPJR/build/asan/bin/kudu with pid 14628
I20260809 22:57:38.245509 15814 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskItYPJR/build/asan/bin/kudu pid 14078
I20260809 22:57:38.246593 14182 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:57:38.352706 15814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskItYPJR/build/asan/bin/kudu with pid 14078
2026-08-09T22:57:38Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (2701 ms)
[----------] 6 tests from EncryptionPolicyTest (21778 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786315935617919-15814-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 22:57:38 dist-test-slave-ncp9 krb5kdc[14883](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 22:57:38 dist-test-slave-ncp9 krb5kdc[14883](info): set up 2 sockets
Aug 09 22:57:38 dist-test-slave-ncp9 krb5kdc[14883](info): commencing operation
krb5kdc: starting...
W20260809 22:57:39.586899 15814 mini_kdc.cc:121] Time spent starting KDC: real 1.181s user 0.001s sys 0.045s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786315935617919-15814-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786315935617919-15814-0/krb5kdc/test-user.keytab.
Aug 09 22:57:39 dist-test-slave-ncp9 krb5kdc[14883](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316259, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T22:57:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T22:57:39Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786315935617919-15814-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786315935617919-15814-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 22:57:39.874323 15814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskItYPJR/build/asan/bin/kudu
/tmp/dist-test-taskItYPJR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786315935617919-15814-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786315935617919-15814-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786315935617919-15814-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786315935617919-15814-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33145
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786315935617919-15814-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37337
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33145
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786315935617919-15814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786315935617919-15814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786315935617919-15814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:57:40.198623 14904 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:57:40.198896 14904 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:57:40.199023 14904 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:57:40.206274 14904 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 22:57:40.206378 14904 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:57:40.206447 14904 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:57:40.206527 14904 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 22:57:40.206589 14904 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 22:57:40.215997 14904 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37337
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786315935617919-15814-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786315935617919-15814-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33145
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786315935617919-15814-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33145
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786315935617919-15814-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786315935617919-15814-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:57:40.217267 14904 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:57:40.218489 14904 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:57:40.229641 14910 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:57:40.229719 14911 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:57:40.229681 14918 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:57:40.234196 14904 server_base.cc:1034] running on GCE node
I20260809 22:57:40.234915 14904 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:57:40.236213 14904 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:57:40.237403 14904 hybrid_clock.cc:648] HybridClock initialized: now 1786316260237358 us; error 39 us; skew 500 ppm
I20260809 22:57:40.237741 14904 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:57:40 dist-test-slave-ncp9 krb5kdc[14883](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316260, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:57:40.240671 14904 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 22:57:40.242153 14904 webserver.cc:460] Webserver started at http://127.0.0.1:44391/ using document root <none> and password file <none>
I20260809 22:57:40.242841 14904 fs_manager.cc:359] Metadata directory not provided
I20260809 22:57:40.242959 14904 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:57:40.243240 14904 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:57:40.246133 14904 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786315935617919-15814-0/minicluster-data/master-0/data/instance:
uuid: "121f6d04c21a4c42b8e3c70d351d1427"
format_stamp: "Formatted at 2026-08-09 22:57:40 on dist-test-slave-ncp9"
server_key: "7961a5c0e57f611238dce56871966e66"
server_key_iv: "2b0f88ca198e60cbf70fede91481fc4b"
server_key_version: "encryptionkey@0"
I20260809 22:57:40.246869 14904 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786315935617919-15814-0/minicluster-data/master-0/wal/instance:
uuid: "121f6d04c21a4c42b8e3c70d351d1427"
format_stamp: "Formatted at 2026-08-09 22:57:40 on dist-test-slave-ncp9"
server_key: "7961a5c0e57f611238dce56871966e66"
server_key_iv: "2b0f88ca198e60cbf70fede91481fc4b"
server_key_version: "encryptionkey@0"
I20260809 22:57:40.251621 14904 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.003s
I20260809 22:57:40.254805 14927 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:57:40.256006 14904 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 22:57:40.256167 14904 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786315935617919-15814-0/minicluster-data/master-0/data,/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786315935617919-15814-0/minicluster-data/master-0/wal
uuid: "121f6d04c21a4c42b8e3c70d351d1427"
format_stamp: "Formatted at 2026-08-09 22:57:40 on dist-test-slave-ncp9"
server_key: "7961a5c0e57f611238dce56871966e66"
server_key_iv: "2b0f88ca198e60cbf70fede91481fc4b"
server_key_version: "encryptionkey@0"
I20260809 22:57:40.256338 14904 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786315935617919-15814-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786315935617919-15814-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786315935617919-15814-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:57:40.284691 14904 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:57:40.288393 14904 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:57:40.288658 14904 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:57:40.305603 14904 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33145
I20260809 22:57:40.305642 15011 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33145 every 8 connection(s)
I20260809 22:57:40.306846 14904 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786315935617919-15814-0/minicluster-data/master-0/data/info.pb
I20260809 22:57:40.307893 15814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskItYPJR/build/asan/bin/kudu as pid 14904
I20260809 22:57:40.308140 15814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786315935617919-15814-0/minicluster-data/master-0/wal/instance
I20260809 22:57:40.308962 15814 external_mini_cluster.cc:1428] Setting key 534b8feacf554b3812f6cf425bbc444c
I20260809 22:57:40.311340 15012 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 09 22:57:40 dist-test-slave-ncp9 krb5kdc[14883](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316259, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260809 22:57:40.323925 15012 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 121f6d04c21a4c42b8e3c70d351d1427: Bootstrap starting.
I20260809 22:57:40.327008 15012 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 121f6d04c21a4c42b8e3c70d351d1427: Neither blocks nor log segments found. Creating new log.
I20260809 22:57:40.328264 15012 log.cc:824] T 00000000000000000000000000000000 P 121f6d04c21a4c42b8e3c70d351d1427: Log is configured to *not* fsync() on all Append() calls
I20260809 22:57:40.331084 15015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:57:40.310842 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34075 (local address 127.0.0.1:33145)
0809 22:57:40.311594 (+ 752us) server_negotiation.cc:206] Beginning negotiation
0809 22:57:40.311605 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 22:57:40.311647 (+ 42us) server_negotiation.cc:407] Connection header received
0809 22:57:40.313257 (+ 1610us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:57:40.313278 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:57:40.313633 (+ 355us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:57:40.313954 (+ 321us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:57:40.315677 (+ 1723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:57:40.317009 (+ 1332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:57:40.318730 (+ 1721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:57:40.319368 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:57:40.319488 (+ 120us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:57:40.324115 (+ 4627us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:57:40.324123 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:57:40.324136 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:57:40.324181 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:57:40.327024 (+ 2843us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:57:40.327895 (+ 871us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:57:40.327901 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:57:40.327906 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:57:40.328070 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:57:40.328576 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:57:40.328580 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:57:40.328582 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:57:40.329257 (+ 675us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:57:40.329582 (+ 325us) server_negotiation.cc:1035] Waiting for connection context
0809 22:57:40.329897 (+ 315us) server_negotiation.cc:299] Negotiation successful
0809 22:57:40.330113 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":551,"thread_start_us":331,"threads_started":1}
I20260809 22:57:40.331305 15012 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 121f6d04c21a4c42b8e3c70d351d1427: No bootstrap required, opened a new log
I20260809 22:57:40.334713 15814 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 22:57:40.340509 15012 raft_consensus.cc:348] T 00000000000000000000000000000000 P 121f6d04c21a4c42b8e3c70d351d1427 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "121f6d04c21a4c42b8e3c70d351d1427" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33145 } }
I20260809 22:57:40.340876 15012 raft_consensus.cc:374] T 00000000000000000000000000000000 P 121f6d04c21a4c42b8e3c70d351d1427 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:57:40.340994 15012 raft_consensus.cc:729] T 00000000000000000000000000000000 P 121f6d04c21a4c42b8e3c70d351d1427 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 121f6d04c21a4c42b8e3c70d351d1427, State: Initialized, Role: FOLLOWER
I20260809 22:57:40.341630 15012 consensus_queue.cc:260] T 00000000000000000000000000000000 P 121f6d04c21a4c42b8e3c70d351d1427 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "121f6d04c21a4c42b8e3c70d351d1427" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33145 } }
I20260809 22:57:40.341861 15012 raft_consensus.cc:388] T 00000000000000000000000000000000 P 121f6d04c21a4c42b8e3c70d351d1427 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 22:57:40.341988 15012 raft_consensus.cc:482] T 00000000000000000000000000000000 P 121f6d04c21a4c42b8e3c70d351d1427 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 22:57:40.342145 15012 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 121f6d04c21a4c42b8e3c70d351d1427 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:57:40.344755 15012 raft_consensus.cc:504] T 00000000000000000000000000000000 P 121f6d04c21a4c42b8e3c70d351d1427 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "121f6d04c21a4c42b8e3c70d351d1427" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33145 } }
I20260809 22:57:40.345240 15012 leader_election.cc:302] T 00000000000000000000000000000000 P 121f6d04c21a4c42b8e3c70d351d1427 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 121f6d04c21a4c42b8e3c70d351d1427; no voters:
I20260809 22:57:40.345821 15012 leader_election.cc:288] T 00000000000000000000000000000000 P 121f6d04c21a4c42b8e3c70d351d1427 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 22:57:40.345958 15026 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 121f6d04c21a4c42b8e3c70d351d1427 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 22:57:40.346633 15026 raft_consensus.cc:686] T 00000000000000000000000000000000 P 121f6d04c21a4c42b8e3c70d351d1427 [term 1 LEADER]: Becoming Leader. State: Replica: 121f6d04c21a4c42b8e3c70d351d1427, State: Running, Role: LEADER
I20260809 22:57:40.347312 15026 consensus_queue.cc:237] T 00000000000000000000000000000000 P 121f6d04c21a4c42b8e3c70d351d1427 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "121f6d04c21a4c42b8e3c70d351d1427" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33145 } }
I20260809 22:57:40.349114 15012 sys_catalog.cc:564] T 00000000000000000000000000000000 P 121f6d04c21a4c42b8e3c70d351d1427 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 22:57:40.349481 15027 sys_catalog.cc:455] T 00000000000000000000000000000000 P 121f6d04c21a4c42b8e3c70d351d1427 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 121f6d04c21a4c42b8e3c70d351d1427. Latest consensus state: current_term: 1 leader_uuid: "121f6d04c21a4c42b8e3c70d351d1427" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "121f6d04c21a4c42b8e3c70d351d1427" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33145 } } }
I20260809 22:57:40.349936 15027 sys_catalog.cc:458] T 00000000000000000000000000000000 P 121f6d04c21a4c42b8e3c70d351d1427 [sys.catalog]: This master's current role is: LEADER
I20260809 22:57:40.351265 15028 sys_catalog.cc:455] T 00000000000000000000000000000000 P 121f6d04c21a4c42b8e3c70d351d1427 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "121f6d04c21a4c42b8e3c70d351d1427" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "121f6d04c21a4c42b8e3c70d351d1427" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33145 } } }
I20260809 22:57:40.351908 15028 sys_catalog.cc:458] T 00000000000000000000000000000000 P 121f6d04c21a4c42b8e3c70d351d1427 [sys.catalog]: This master's current role is: LEADER
I20260809 22:57:40.357741 15015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:57:40.340376 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59575 (local address 127.0.0.1:33145)
0809 22:57:40.340559 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0809 22:57:40.340564 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:57:40.341021 (+ 457us) server_negotiation.cc:407] Connection header received
0809 22:57:40.341207 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:57:40.341211 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:57:40.341283 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:57:40.341396 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:57:40.343011 (+ 1615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:57:40.344135 (+ 1124us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:57:40.345533 (+ 1398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:57:40.346276 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:57:40.346409 (+ 133us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:57:40.349627 (+ 3218us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:57:40.349635 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:57:40.349639 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:57:40.349675 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:57:40.353429 (+ 3754us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:57:40.354460 (+ 1031us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:57:40.354467 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:57:40.354471 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:57:40.354676 (+ 205us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:57:40.355314 (+ 638us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:57:40.355319 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:57:40.355323 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:57:40.355714 (+ 391us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:57:40.356008 (+ 294us) server_negotiation.cc:1035] Waiting for connection context
0809 22:57:40.357157 (+ 1149us) server_negotiation.cc:299] Negotiation successful
0809 22:57:40.357352 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260809 22:57:40.360252 15030 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 22:57:40.365870 15030 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 22:57:40.374130 15030 catalog_manager.cc:1347] Generated new cluster ID: 93632e7c50fd47479d9afc6856e55a41
I20260809 22:57:40.374236 15030 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 22:57:40.390024 15030 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 22:57:40.391211 15030 catalog_manager.cc:1504] Loading token signing keys...
I20260809 22:57:40.396373 15030 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 121f6d04c21a4c42b8e3c70d351d1427: Generated new TSK 0
I20260809 22:57:40.397142 15030 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 22:57:40.408317 15814 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskItYPJR/build/asan/bin/kudu pid 14904
I20260809 22:57:40.409559 15005 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:57:40.498162 15814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskItYPJR/build/asan/bin/kudu with pid 14904
W20260809 22:57:40.526680 14881 connection.cc:535] client connection to 127.0.0.1:33145 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:33145): Connection reset by peer (error 104)
2026-08-09T22:57:40Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (2139 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786315935617919-15814-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 22:57:40 dist-test-slave-ncp9 krb5kdc[15058](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 22:57:40 dist-test-slave-ncp9 krb5kdc[15058](info): set up 2 sockets
Aug 09 22:57:40 dist-test-slave-ncp9 krb5kdc[15058](info): commencing operation
krb5kdc: starting...
W20260809 22:57:41.723814 15814 mini_kdc.cc:121] Time spent starting KDC: real 1.178s user 0.002s sys 0.041s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786315935617919-15814-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786315935617919-15814-0/krb5kdc/test-user.keytab.
Aug 09 22:57:41 dist-test-slave-ncp9 krb5kdc[15058](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316261, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T22:57:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T22:57:41Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786315935617919-15814-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786315935617919-15814-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 22:57:42.011885 15814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskItYPJR/build/asan/bin/kudu
/tmp/dist-test-taskItYPJR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786315935617919-15814-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786315935617919-15814-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786315935617919-15814-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786315935617919-15814-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39993
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786315935617919-15814-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43647
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39993
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786315935617919-15814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786315935617919-15814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786315935617919-15814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:57:42.345158 15079 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:57:42.345463 15079 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:57:42.345607 15079 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:57:42.352484 15079 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 22:57:42.352595 15079 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:57:42.352672 15079 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:57:42.352756 15079 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 22:57:42.352818 15079 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 22:57:42.362214 15079 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43647
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786315935617919-15814-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786315935617919-15814-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39993
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786315935617919-15814-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39993
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786315935617919-15814-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786315935617919-15814-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:57:42.363407 15079 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:57:42.364670 15079 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:57:42.375691 15089 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:57:42.375653 15092 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:57:42.375847 15094 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:57:42.376386 15079 server_base.cc:1034] running on GCE node
I20260809 22:57:42.377099 15079 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:57:42.378363 15079 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:57:42.379557 15079 hybrid_clock.cc:648] HybridClock initialized: now 1786316262379505 us; error 42 us; skew 500 ppm
I20260809 22:57:42.379943 15079 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:57:42 dist-test-slave-ncp9 krb5kdc[15058](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316262, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:57:42.385766 15079 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 22:57:42.387092 15079 webserver.cc:460] Webserver started at http://127.0.0.1:46813/ using document root <none> and password file <none>
I20260809 22:57:42.387724 15079 fs_manager.cc:359] Metadata directory not provided
I20260809 22:57:42.387847 15079 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:57:42.388131 15079 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:57:42.390992 15079 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786315935617919-15814-0/minicluster-data/master-0/data/instance:
uuid: "55344bb15fbb4d398d2b70d3835a48a0"
format_stamp: "Formatted at 2026-08-09 22:57:42 on dist-test-slave-ncp9"
server_key: "29bb0b3c0ee09a035e6c4449bb0987fc"
server_key_iv: "81f28820585a4ad65cad55f53ab183d2"
server_key_version: "encryptionkey@0"
I20260809 22:57:42.391744 15079 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786315935617919-15814-0/minicluster-data/master-0/wal/instance:
uuid: "55344bb15fbb4d398d2b70d3835a48a0"
format_stamp: "Formatted at 2026-08-09 22:57:42 on dist-test-slave-ncp9"
server_key: "29bb0b3c0ee09a035e6c4449bb0987fc"
server_key_iv: "81f28820585a4ad65cad55f53ab183d2"
server_key_version: "encryptionkey@0"
I20260809 22:57:42.396575 15079 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.004s
I20260809 22:57:42.399806 15107 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:57:42.401044 15079 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.002s
I20260809 22:57:42.401208 15079 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786315935617919-15814-0/minicluster-data/master-0/data,/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786315935617919-15814-0/minicluster-data/master-0/wal
uuid: "55344bb15fbb4d398d2b70d3835a48a0"
format_stamp: "Formatted at 2026-08-09 22:57:42 on dist-test-slave-ncp9"
server_key: "29bb0b3c0ee09a035e6c4449bb0987fc"
server_key_iv: "81f28820585a4ad65cad55f53ab183d2"
server_key_version: "encryptionkey@0"
I20260809 22:57:42.401377 15079 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786315935617919-15814-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786315935617919-15814-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786315935617919-15814-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:57:42.427210 15079 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:57:42.430949 15079 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:57:42.431234 15079 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:57:42.448254 15079 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39993
I20260809 22:57:42.448287 15181 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39993 every 8 connection(s)
I20260809 22:57:42.449548 15079 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786315935617919-15814-0/minicluster-data/master-0/data/info.pb
I20260809 22:57:42.453051 15183 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:57:42.459260 15814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskItYPJR/build/asan/bin/kudu as pid 15079
I20260809 22:57:42.459458 15814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786315935617919-15814-0/minicluster-data/master-0/wal/instance
I20260809 22:57:42.459991 15814 external_mini_cluster.cc:1428] Setting key 0391211624cab02974466e639123add6
I20260809 22:57:42.463529 15183 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 55344bb15fbb4d398d2b70d3835a48a0: Bootstrap starting.
I20260809 22:57:42.466693 15183 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 55344bb15fbb4d398d2b70d3835a48a0: Neither blocks nor log segments found. Creating new log.
I20260809 22:57:42.467890 15183 log.cc:824] T 00000000000000000000000000000000 P 55344bb15fbb4d398d2b70d3835a48a0: Log is configured to *not* fsync() on all Append() calls
I20260809 22:57:42.471010 15183 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 55344bb15fbb4d398d2b70d3835a48a0: No bootstrap required, opened a new log
I20260809 22:57:42.473222 15187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:57:42.461766 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57359 (local address 127.0.0.1:39993)
0809 22:57:42.462485 (+ 719us) server_negotiation.cc:206] Beginning negotiation
0809 22:57:42.462496 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 22:57:42.462535 (+ 39us) server_negotiation.cc:407] Connection header received
0809 22:57:42.464183 (+ 1648us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:57:42.464206 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:57:42.464567 (+ 361us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:57:42.464928 (+ 361us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:57:42.465525 (+ 597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:57:42.466794 (+ 1269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:57:42.468602 (+ 1808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:57:42.469511 (+ 909us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:57:42.469642 (+ 131us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:57:42.470977 (+ 1335us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:57:42.470985 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:57:42.471002 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 22:57:42.471540 (+ 538us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:57:42.471611 (+ 71us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 22:57:42.471638 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:57:42.471761 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0809 22:57:42.472005 (+ 244us) server_negotiation.cc:299] Negotiation successful
0809 22:57:42.472191 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":527,"thread_start_us":338,"threads_started":1}
I20260809 22:57:42.476876 15814 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 22:57:42.479425 15183 raft_consensus.cc:348] T 00000000000000000000000000000000 P 55344bb15fbb4d398d2b70d3835a48a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55344bb15fbb4d398d2b70d3835a48a0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39993 } }
I20260809 22:57:42.479863 15183 raft_consensus.cc:374] T 00000000000000000000000000000000 P 55344bb15fbb4d398d2b70d3835a48a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:57:42.479970 15183 raft_consensus.cc:729] T 00000000000000000000000000000000 P 55344bb15fbb4d398d2b70d3835a48a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 55344bb15fbb4d398d2b70d3835a48a0, State: Initialized, Role: FOLLOWER
I20260809 22:57:42.480660 15183 consensus_queue.cc:260] T 00000000000000000000000000000000 P 55344bb15fbb4d398d2b70d3835a48a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55344bb15fbb4d398d2b70d3835a48a0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39993 } }
I20260809 22:57:42.480973 15183 raft_consensus.cc:388] T 00000000000000000000000000000000 P 55344bb15fbb4d398d2b70d3835a48a0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 22:57:42.481109 15183 raft_consensus.cc:482] T 00000000000000000000000000000000 P 55344bb15fbb4d398d2b70d3835a48a0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 22:57:42.481294 15183 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 55344bb15fbb4d398d2b70d3835a48a0 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:57:42.484216 15183 raft_consensus.cc:504] T 00000000000000000000000000000000 P 55344bb15fbb4d398d2b70d3835a48a0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55344bb15fbb4d398d2b70d3835a48a0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39993 } }
I20260809 22:57:42.484750 15183 leader_election.cc:302] T 00000000000000000000000000000000 P 55344bb15fbb4d398d2b70d3835a48a0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 55344bb15fbb4d398d2b70d3835a48a0; no voters:
I20260809 22:57:42.485453 15183 leader_election.cc:288] T 00000000000000000000000000000000 P 55344bb15fbb4d398d2b70d3835a48a0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 22:57:42.485539 15198 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 55344bb15fbb4d398d2b70d3835a48a0 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 22:57:42.486716 15198 raft_consensus.cc:686] T 00000000000000000000000000000000 P 55344bb15fbb4d398d2b70d3835a48a0 [term 1 LEADER]: Becoming Leader. State: Replica: 55344bb15fbb4d398d2b70d3835a48a0, State: Running, Role: LEADER
I20260809 22:57:42.487260 15198 consensus_queue.cc:237] T 00000000000000000000000000000000 P 55344bb15fbb4d398d2b70d3835a48a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55344bb15fbb4d398d2b70d3835a48a0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39993 } }
I20260809 22:57:42.487691 15183 sys_catalog.cc:564] T 00000000000000000000000000000000 P 55344bb15fbb4d398d2b70d3835a48a0 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 22:57:42.489595 15187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:57:42.482268 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34695 (local address 127.0.0.1:39993)
0809 22:57:42.482449 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0809 22:57:42.482454 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:57:42.482710 (+ 256us) server_negotiation.cc:407] Connection header received
0809 22:57:42.482865 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:57:42.482868 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:57:42.482929 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:57:42.483022 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:57:42.483661 (+ 639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:57:42.484425 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:57:42.486312 (+ 1887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:57:42.487008 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:57:42.487104 (+ 96us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:57:42.488597 (+ 1493us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:57:42.488605 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:57:42.488610 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 22:57:42.488633 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:57:42.488711 (+ 78us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 22:57:42.488733 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:57:42.488858 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0809 22:57:42.489217 (+ 359us) server_negotiation.cc:299] Negotiation successful
0809 22:57:42.489279 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260809 22:57:42.489989 15199 sys_catalog.cc:455] T 00000000000000000000000000000000 P 55344bb15fbb4d398d2b70d3835a48a0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "55344bb15fbb4d398d2b70d3835a48a0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55344bb15fbb4d398d2b70d3835a48a0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39993 } } }
I20260809 22:57:42.490348 15199 sys_catalog.cc:458] T 00000000000000000000000000000000 P 55344bb15fbb4d398d2b70d3835a48a0 [sys.catalog]: This master's current role is: LEADER
I20260809 22:57:42.491431 15200 sys_catalog.cc:455] T 00000000000000000000000000000000 P 55344bb15fbb4d398d2b70d3835a48a0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 55344bb15fbb4d398d2b70d3835a48a0. Latest consensus state: current_term: 1 leader_uuid: "55344bb15fbb4d398d2b70d3835a48a0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55344bb15fbb4d398d2b70d3835a48a0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39993 } } }
I20260809 22:57:42.491781 15200 sys_catalog.cc:458] T 00000000000000000000000000000000 P 55344bb15fbb4d398d2b70d3835a48a0 [sys.catalog]: This master's current role is: LEADER
I20260809 22:57:42.492750 15203 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 22:57:42.497592 15203 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 22:57:42.505326 15203 catalog_manager.cc:1347] Generated new cluster ID: d0b488b4ff4a4f31aec74b1fafc92add
I20260809 22:57:42.505429 15203 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 22:57:42.514565 15203 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 22:57:42.516369 15203 catalog_manager.cc:1504] Loading token signing keys...
I20260809 22:57:42.526669 15203 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 55344bb15fbb4d398d2b70d3835a48a0: Generated new TSK 0
I20260809 22:57:42.527621 15203 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 22:57:42.534809 15814 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskItYPJR/build/asan/bin/kudu pid 15079
W20260809 22:57:42.535815 15176 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:57359
W20260809 22:57:42.536520 15814 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskItYPJR/build/asan/bin/kudu pid 15079: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 22:57:42.536728 15814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskItYPJR/build/asan/bin/kudu with pid 15079
W20260809 22:57:42.562361 15056 connection.cc:535] client connection to 127.0.0.1:39993 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:39993): Connection reset by peer (error 104)
2026-08-09T22:57:42Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (2035 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786315935617919-15814-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 22:57:42 dist-test-slave-ncp9 krb5kdc[15233](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 22:57:42 dist-test-slave-ncp9 krb5kdc[15233](info): set up 2 sockets
Aug 09 22:57:42 dist-test-slave-ncp9 krb5kdc[15233](info): commencing operation
krb5kdc: starting...
W20260809 22:57:43.778859 15814 mini_kdc.cc:121] Time spent starting KDC: real 1.198s user 0.003s sys 0.040s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786315935617919-15814-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786315935617919-15814-0/krb5kdc/test-user.keytab.
Aug 09 22:57:43 dist-test-slave-ncp9 krb5kdc[15233](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316263, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T22:57:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T22:57:44Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786315935617919-15814-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786315935617919-15814-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 22:57:44.076264 15814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskItYPJR/build/asan/bin/kudu
/tmp/dist-test-taskItYPJR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786315935617919-15814-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786315935617919-15814-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786315935617919-15814-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786315935617919-15814-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33889
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786315935617919-15814-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34437
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33889
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786315935617919-15814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786315935617919-15814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786315935617919-15814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:57:44.411522 15249 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:57:44.411782 15249 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:57:44.411885 15249 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:57:44.418725 15249 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 22:57:44.418819 15249 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:57:44.418870 15249 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:57:44.418929 15249 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 22:57:44.418972 15249 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 22:57:44.428207 15249 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34437
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786315935617919-15814-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786315935617919-15814-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33889
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786315935617919-15814-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33889
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786315935617919-15814-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786315935617919-15814-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:57:44.429317 15249 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:57:44.430523 15249 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:57:44.441449 15261 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:57:44.441480 15263 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:57:44.441515 15265 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:57:44.442350 15249 server_base.cc:1034] running on GCE node
I20260809 22:57:44.443071 15249 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:57:44.444319 15249 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:57:44.445504 15249 hybrid_clock.cc:648] HybridClock initialized: now 1786316264445456 us; error 35 us; skew 500 ppm
I20260809 22:57:44.445837 15249 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:57:44 dist-test-slave-ncp9 krb5kdc[15233](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316264, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:57:44.448756 15249 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 22:57:44.450107 15249 webserver.cc:460] Webserver started at http://127.0.0.1:43659/ using document root <none> and password file <none>
I20260809 22:57:44.450726 15249 fs_manager.cc:359] Metadata directory not provided
I20260809 22:57:44.450857 15249 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:57:44.451139 15249 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:57:44.454030 15249 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786315935617919-15814-0/minicluster-data/master-0/data/instance:
uuid: "9f2cd28d9c61494bb3e122a400803191"
format_stamp: "Formatted at 2026-08-09 22:57:44 on dist-test-slave-ncp9"
server_key: "61c4459a3298ce02c2495678ce0994a1"
server_key_iv: "c8b2878f3310937067c1cf99340c6200"
server_key_version: "encryptionkey@0"
I20260809 22:57:44.454747 15249 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786315935617919-15814-0/minicluster-data/master-0/wal/instance:
uuid: "9f2cd28d9c61494bb3e122a400803191"
format_stamp: "Formatted at 2026-08-09 22:57:44 on dist-test-slave-ncp9"
server_key: "61c4459a3298ce02c2495678ce0994a1"
server_key_iv: "c8b2878f3310937067c1cf99340c6200"
server_key_version: "encryptionkey@0"
I20260809 22:57:44.459563 15249 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.003s
I20260809 22:57:44.462826 15278 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:57:44.464031 15249 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 22:57:44.464181 15249 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786315935617919-15814-0/minicluster-data/master-0/data,/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786315935617919-15814-0/minicluster-data/master-0/wal
uuid: "9f2cd28d9c61494bb3e122a400803191"
format_stamp: "Formatted at 2026-08-09 22:57:44 on dist-test-slave-ncp9"
server_key: "61c4459a3298ce02c2495678ce0994a1"
server_key_iv: "c8b2878f3310937067c1cf99340c6200"
server_key_version: "encryptionkey@0"
I20260809 22:57:44.464355 15249 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786315935617919-15814-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786315935617919-15814-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786315935617919-15814-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:57:44.469852 15249 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:57:44.473489 15249 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:57:44.473752 15249 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:57:44.491010 15249 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33889
I20260809 22:57:44.491060 15355 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33889 every 8 connection(s)
I20260809 22:57:44.492478 15249 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786315935617919-15814-0/minicluster-data/master-0/data/info.pb
I20260809 22:57:44.496325 15358 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:57:44.501895 15814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskItYPJR/build/asan/bin/kudu as pid 15249
I20260809 22:57:44.502085 15814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786315935617919-15814-0/minicluster-data/master-0/wal/instance
I20260809 22:57:44.502651 15814 external_mini_cluster.cc:1428] Setting key 4bee6fb018b2e428e8637c52e423be8b
I20260809 22:57:44.507488 15358 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9f2cd28d9c61494bb3e122a400803191: Bootstrap starting.
I20260809 22:57:44.510443 15361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:57:44.504479 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49937 (local address 127.0.0.1:33889)
0809 22:57:44.505362 (+ 883us) server_negotiation.cc:206] Beginning negotiation
0809 22:57:44.505374 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 22:57:44.505416 (+ 42us) server_negotiation.cc:407] Connection header received
0809 22:57:44.507050 (+ 1634us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:57:44.507074 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:57:44.507417 (+ 343us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:57:44.507754 (+ 337us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:57:44.508774 (+ 1020us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:57:44.508780 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:57:44.508794 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 22:57:44.508808 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:57:44.508869 (+ 61us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 22:57:44.508918 (+ 49us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:57:44.509103 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0809 22:57:44.509233 (+ 130us) server_negotiation.cc:299] Negotiation successful
0809 22:57:44.509434 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":646,"thread_start_us":415,"threads_started":1}
I20260809 22:57:44.511247 15358 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9f2cd28d9c61494bb3e122a400803191: Neither blocks nor log segments found. Creating new log.
I20260809 22:57:44.512382 15358 log.cc:824] T 00000000000000000000000000000000 P 9f2cd28d9c61494bb3e122a400803191: Log is configured to *not* fsync() on all Append() calls
I20260809 22:57:44.514015 15814 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 22:57:44.515182 15358 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9f2cd28d9c61494bb3e122a400803191: No bootstrap required, opened a new log
I20260809 22:57:44.520996 15361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:57:44.518354 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58435 (local address 127.0.0.1:33889)
0809 22:57:44.518495 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0809 22:57:44.518500 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:57:44.519042 (+ 542us) server_negotiation.cc:407] Connection header received
0809 22:57:44.519206 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:57:44.519211 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:57:44.519259 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:57:44.519337 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:57:44.520304 (+ 967us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:57:44.520307 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:57:44.520310 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 22:57:44.520324 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:57:44.520364 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 22:57:44.520378 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:57:44.520480 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0809 22:57:44.520637 (+ 157us) server_negotiation.cc:299] Negotiation successful
0809 22:57:44.520699 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260809 22:57:44.524271 15358 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9f2cd28d9c61494bb3e122a400803191 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f2cd28d9c61494bb3e122a400803191" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33889 } }
I20260809 22:57:44.524695 15358 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9f2cd28d9c61494bb3e122a400803191 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:57:44.524798 15358 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9f2cd28d9c61494bb3e122a400803191 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9f2cd28d9c61494bb3e122a400803191, State: Initialized, Role: FOLLOWER
I20260809 22:57:44.525477 15358 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9f2cd28d9c61494bb3e122a400803191 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f2cd28d9c61494bb3e122a400803191" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33889 } }
I20260809 22:57:44.525727 15358 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9f2cd28d9c61494bb3e122a400803191 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 22:57:44.525872 15358 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9f2cd28d9c61494bb3e122a400803191 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 22:57:44.526042 15358 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9f2cd28d9c61494bb3e122a400803191 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:57:44.528400 15358 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9f2cd28d9c61494bb3e122a400803191 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f2cd28d9c61494bb3e122a400803191" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33889 } }
I20260809 22:57:44.528967 15358 leader_election.cc:302] T 00000000000000000000000000000000 P 9f2cd28d9c61494bb3e122a400803191 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9f2cd28d9c61494bb3e122a400803191; no voters:
I20260809 22:57:44.529561 15358 leader_election.cc:288] T 00000000000000000000000000000000 P 9f2cd28d9c61494bb3e122a400803191 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 22:57:44.529675 15372 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9f2cd28d9c61494bb3e122a400803191 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 22:57:44.530361 15372 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9f2cd28d9c61494bb3e122a400803191 [term 1 LEADER]: Becoming Leader. State: Replica: 9f2cd28d9c61494bb3e122a400803191, State: Running, Role: LEADER
I20260809 22:57:44.530946 15372 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9f2cd28d9c61494bb3e122a400803191 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f2cd28d9c61494bb3e122a400803191" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33889 } }
I20260809 22:57:44.531759 15358 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9f2cd28d9c61494bb3e122a400803191 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 22:57:44.533771 15374 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9f2cd28d9c61494bb3e122a400803191 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9f2cd28d9c61494bb3e122a400803191" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f2cd28d9c61494bb3e122a400803191" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33889 } } }
I20260809 22:57:44.534113 15374 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9f2cd28d9c61494bb3e122a400803191 [sys.catalog]: This master's current role is: LEADER
I20260809 22:57:44.534754 15375 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9f2cd28d9c61494bb3e122a400803191 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9f2cd28d9c61494bb3e122a400803191. Latest consensus state: current_term: 1 leader_uuid: "9f2cd28d9c61494bb3e122a400803191" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f2cd28d9c61494bb3e122a400803191" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33889 } } }
I20260809 22:57:44.535089 15375 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9f2cd28d9c61494bb3e122a400803191 [sys.catalog]: This master's current role is: LEADER
I20260809 22:57:44.538194 15382 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 22:57:44.544536 15382 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 22:57:44.553694 15382 catalog_manager.cc:1347] Generated new cluster ID: faa5460302d0437f9da8a201253c77d7
I20260809 22:57:44.553821 15382 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 22:57:44.572479 15382 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 22:57:44.572983 15382 catalog_manager.cc:1504] Loading token signing keys...
I20260809 22:57:44.581481 15382 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9f2cd28d9c61494bb3e122a400803191: Generated new TSK 0
I20260809 22:57:44.582214 15382 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 22:57:44.596747 15814 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskItYPJR/build/asan/bin/kudu pid 15249
W20260809 22:57:44.597777 15347 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:49937
W20260809 22:57:44.598402 15814 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskItYPJR/build/asan/bin/kudu pid 15249: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 22:57:44.598531 15814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskItYPJR/build/asan/bin/kudu with pid 15249
2026-08-09T22:57:44Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (2063 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786315935617919-15814-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 22:57:44 dist-test-slave-ncp9 krb5kdc[15403](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 22:57:44 dist-test-slave-ncp9 krb5kdc[15403](info): set up 2 sockets
Aug 09 22:57:44 dist-test-slave-ncp9 krb5kdc[15403](info): commencing operation
krb5kdc: starting...
W20260809 22:57:45.829187 15814 mini_kdc.cc:121] Time spent starting KDC: real 1.185s user 0.003s sys 0.043s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786315935617919-15814-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786315935617919-15814-0/krb5kdc/test-user.keytab.
Aug 09 22:57:46 dist-test-slave-ncp9 krb5kdc[15403](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316266, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T22:57:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T22:57:46Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786315935617919-15814-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786315935617919-15814-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 22:57:46.130259 15814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskItYPJR/build/asan/bin/kudu
/tmp/dist-test-taskItYPJR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786315935617919-15814-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786315935617919-15814-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786315935617919-15814-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786315935617919-15814-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42507
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786315935617919-15814-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37277
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42507
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786315935617919-15814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786315935617919-15814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786315935617919-15814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:57:46.456413 15426 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:57:46.456688 15426 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:57:46.456825 15426 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:57:46.463737 15426 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 22:57:46.463846 15426 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:57:46.463922 15426 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:57:46.464004 15426 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 22:57:46.464062 15426 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 22:57:46.473359 15426 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37277
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786315935617919-15814-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786315935617919-15814-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42507
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786315935617919-15814-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42507
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786315935617919-15814-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786315935617919-15814-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:57:46.474514 15426 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:57:46.475795 15426 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:57:46.486758 15434 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:57:46.486759 15441 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:57:46.486799 15435 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:57:46.490067 15426 server_base.cc:1034] running on GCE node
I20260809 22:57:46.490798 15426 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:57:46.492100 15426 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:57:46.493304 15426 hybrid_clock.cc:648] HybridClock initialized: now 1786316266493258 us; error 41 us; skew 500 ppm
I20260809 22:57:46.493631 15426 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:57:46 dist-test-slave-ncp9 krb5kdc[15403](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316266, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:57:46.496630 15426 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 22:57:46.498100 15426 webserver.cc:460] Webserver started at http://127.0.0.1:34843/ using document root <none> and password file <none>
I20260809 22:57:46.498770 15426 fs_manager.cc:359] Metadata directory not provided
I20260809 22:57:46.498885 15426 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:57:46.499176 15426 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:57:46.502228 15426 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786315935617919-15814-0/minicluster-data/master-0/data/instance:
uuid: "4eaa5186902b4a22a625209884d33027"
format_stamp: "Formatted at 2026-08-09 22:57:46 on dist-test-slave-ncp9"
server_key: "f16c4123fda00e2d75f6df0a1a13425c"
server_key_iv: "e7240c252bec4b21a5bd70d49cecec2d"
server_key_version: "encryptionkey@0"
I20260809 22:57:46.502991 15426 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786315935617919-15814-0/minicluster-data/master-0/wal/instance:
uuid: "4eaa5186902b4a22a625209884d33027"
format_stamp: "Formatted at 2026-08-09 22:57:46 on dist-test-slave-ncp9"
server_key: "f16c4123fda00e2d75f6df0a1a13425c"
server_key_iv: "e7240c252bec4b21a5bd70d49cecec2d"
server_key_version: "encryptionkey@0"
I20260809 22:57:46.507967 15426 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.004s
I20260809 22:57:46.511322 15453 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:57:46.512631 15426 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 22:57:46.512785 15426 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786315935617919-15814-0/minicluster-data/master-0/data,/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786315935617919-15814-0/minicluster-data/master-0/wal
uuid: "4eaa5186902b4a22a625209884d33027"
format_stamp: "Formatted at 2026-08-09 22:57:46 on dist-test-slave-ncp9"
server_key: "f16c4123fda00e2d75f6df0a1a13425c"
server_key_iv: "e7240c252bec4b21a5bd70d49cecec2d"
server_key_version: "encryptionkey@0"
I20260809 22:57:46.513010 15426 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786315935617919-15814-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786315935617919-15814-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786315935617919-15814-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:57:46.529004 15426 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:57:46.532552 15426 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:57:46.532800 15426 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:57:46.550010 15426 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42507
I20260809 22:57:46.550040 15533 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42507 every 8 connection(s)
I20260809 22:57:46.551314 15426 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786315935617919-15814-0/minicluster-data/master-0/data/info.pb
I20260809 22:57:46.554976 15534 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:57:46.556959 15814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskItYPJR/build/asan/bin/kudu as pid 15426
I20260809 22:57:46.557145 15814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786315935617919-15814-0/minicluster-data/master-0/wal/instance
I20260809 22:57:46.557716 15814 external_mini_cluster.cc:1428] Setting key db466b09d78a24075fdcf52030396876
I20260809 22:57:46.568652 15534 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4eaa5186902b4a22a625209884d33027: Bootstrap starting.
Aug 09 22:57:46 dist-test-slave-ncp9 krb5kdc[15403](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316266, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260809 22:57:46.572398 15534 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4eaa5186902b4a22a625209884d33027: Neither blocks nor log segments found. Creating new log.
I20260809 22:57:46.573593 15534 log.cc:824] T 00000000000000000000000000000000 P 4eaa5186902b4a22a625209884d33027: Log is configured to *not* fsync() on all Append() calls
I20260809 22:57:46.576841 15534 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4eaa5186902b4a22a625209884d33027: No bootstrap required, opened a new log
I20260809 22:57:46.581151 15537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:57:46.559438 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39593 (local address 127.0.0.1:42507)
0809 22:57:46.560230 (+ 792us) server_negotiation.cc:206] Beginning negotiation
0809 22:57:46.560240 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 22:57:46.560281 (+ 41us) server_negotiation.cc:407] Connection header received
0809 22:57:46.561845 (+ 1564us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:57:46.561870 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:57:46.562256 (+ 386us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:57:46.562543 (+ 287us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:57:46.564345 (+ 1802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:57:46.565686 (+ 1341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:57:46.567435 (+ 1749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:57:46.568080 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:57:46.568199 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:57:46.573089 (+ 4890us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:57:46.573097 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:57:46.573110 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:57:46.573143 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:57:46.576705 (+ 3562us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:57:46.577452 (+ 747us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:57:46.577460 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:57:46.577468 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:57:46.577688 (+ 220us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:57:46.578014 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:57:46.578018 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:57:46.578021 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:57:46.578726 (+ 705us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:57:46.579052 (+ 326us) server_negotiation.cc:1035] Waiting for connection context
0809 22:57:46.579361 (+ 309us) server_negotiation.cc:299] Negotiation successful
0809 22:57:46.579809 (+ 448us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":587,"thread_start_us":385,"threads_started":1}
I20260809 22:57:46.584563 15814 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 22:57:46.586131 15534 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4eaa5186902b4a22a625209884d33027 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4eaa5186902b4a22a625209884d33027" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42507 } }
I20260809 22:57:46.586540 15534 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4eaa5186902b4a22a625209884d33027 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:57:46.586632 15534 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4eaa5186902b4a22a625209884d33027 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4eaa5186902b4a22a625209884d33027, State: Initialized, Role: FOLLOWER
I20260809 22:57:46.587278 15534 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4eaa5186902b4a22a625209884d33027 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4eaa5186902b4a22a625209884d33027" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42507 } }
I20260809 22:57:46.587565 15534 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4eaa5186902b4a22a625209884d33027 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 22:57:46.587719 15534 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4eaa5186902b4a22a625209884d33027 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 22:57:46.587877 15534 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4eaa5186902b4a22a625209884d33027 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:57:46.590633 15534 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4eaa5186902b4a22a625209884d33027 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4eaa5186902b4a22a625209884d33027" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42507 } }
I20260809 22:57:46.591078 15534 leader_election.cc:302] T 00000000000000000000000000000000 P 4eaa5186902b4a22a625209884d33027 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4eaa5186902b4a22a625209884d33027; no voters:
I20260809 22:57:46.591647 15534 leader_election.cc:288] T 00000000000000000000000000000000 P 4eaa5186902b4a22a625209884d33027 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 22:57:46.591763 15551 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4eaa5186902b4a22a625209884d33027 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 22:57:46.593789 15534 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4eaa5186902b4a22a625209884d33027 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 22:57:46.595013 15551 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4eaa5186902b4a22a625209884d33027 [term 1 LEADER]: Becoming Leader. State: Replica: 4eaa5186902b4a22a625209884d33027, State: Running, Role: LEADER
I20260809 22:57:46.595810 15551 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4eaa5186902b4a22a625209884d33027 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4eaa5186902b4a22a625209884d33027" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42507 } }
I20260809 22:57:46.598590 15554 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4eaa5186902b4a22a625209884d33027 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4eaa5186902b4a22a625209884d33027. Latest consensus state: current_term: 1 leader_uuid: "4eaa5186902b4a22a625209884d33027" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4eaa5186902b4a22a625209884d33027" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42507 } } }
I20260809 22:57:46.598992 15554 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4eaa5186902b4a22a625209884d33027 [sys.catalog]: This master's current role is: LEADER
I20260809 22:57:46.604638 15553 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4eaa5186902b4a22a625209884d33027 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4eaa5186902b4a22a625209884d33027" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4eaa5186902b4a22a625209884d33027" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42507 } } }
I20260809 22:57:46.605444 15553 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4eaa5186902b4a22a625209884d33027 [sys.catalog]: This master's current role is: LEADER
I20260809 22:57:46.605918 15537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:57:46.589475 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48971 (local address 127.0.0.1:42507)
0809 22:57:46.589655 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0809 22:57:46.589661 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:57:46.590102 (+ 441us) server_negotiation.cc:407] Connection header received
0809 22:57:46.590210 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:57:46.590215 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:57:46.590283 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:57:46.590383 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:57:46.591698 (+ 1315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:57:46.592731 (+ 1033us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:57:46.594371 (+ 1640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:57:46.594959 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:57:46.595733 (+ 774us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:57:46.598282 (+ 2549us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:57:46.598289 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:57:46.598295 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:57:46.598334 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:57:46.601975 (+ 3641us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:57:46.602840 (+ 865us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:57:46.602846 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:57:46.602850 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:57:46.602995 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:57:46.603432 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:57:46.603437 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:57:46.603441 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:57:46.603839 (+ 398us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:57:46.604095 (+ 256us) server_negotiation.cc:1035] Waiting for connection context
0809 22:57:46.605308 (+ 1213us) server_negotiation.cc:299] Negotiation successful
0809 22:57:46.605542 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":56}
I20260809 22:57:46.609252 15561 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 22:57:46.614169 15561 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 22:57:46.623309 15561 catalog_manager.cc:1347] Generated new cluster ID: 5254e12947c44425bd9939a6fd38fe6b
I20260809 22:57:46.623406 15561 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 22:57:46.636533 15561 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 22:57:46.638314 15561 catalog_manager.cc:1504] Loading token signing keys...
I20260809 22:57:46.647174 15561 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4eaa5186902b4a22a625209884d33027: Generated new TSK 0
I20260809 22:57:46.647953 15561 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 22:57:46.660477 15814 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskItYPJR/build/asan/bin/kudu pid 15426
I20260809 22:57:46.661604 15525 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:57:46.750885 15814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskItYPJR/build/asan/bin/kudu with pid 15426
2026-08-09T22:57:46Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (2151 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786315935617919-15814-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 22:57:46 dist-test-slave-ncp9 krb5kdc[15581](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 22:57:46 dist-test-slave-ncp9 krb5kdc[15581](info): set up 2 sockets
Aug 09 22:57:46 dist-test-slave-ncp9 krb5kdc[15581](info): commencing operation
krb5kdc: starting...
W20260809 22:57:47.998615 15814 mini_kdc.cc:121] Time spent starting KDC: real 1.203s user 0.004s sys 0.039s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786315935617919-15814-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786315935617919-15814-0/krb5kdc/test-user.keytab.
Aug 09 22:57:48 dist-test-slave-ncp9 krb5kdc[15581](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316268, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T22:57:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T22:57:48Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786315935617919-15814-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786315935617919-15814-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 22:57:48.286279 15814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskItYPJR/build/asan/bin/kudu
/tmp/dist-test-taskItYPJR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786315935617919-15814-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786315935617919-15814-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786315935617919-15814-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786315935617919-15814-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45595
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786315935617919-15814-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44243
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45595
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786315935617919-15814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786315935617919-15814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786315935617919-15814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:57:48.611896 15598 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:57:48.612185 15598 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:57:48.612316 15598 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:57:48.619659 15598 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 22:57:48.619797 15598 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:57:48.619879 15598 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:57:48.619966 15598 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 22:57:48.620024 15598 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 22:57:48.629691 15598 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44243
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786315935617919-15814-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786315935617919-15814-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45595
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786315935617919-15814-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45595
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786315935617919-15814-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786315935617919-15814-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:57:48.630897 15598 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:57:48.632181 15598 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:57:48.643354 15612 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:57:48.643250 15616 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:57:48.643397 15614 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:57:48.644356 15598 server_base.cc:1034] running on GCE node
I20260809 22:57:48.645105 15598 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:57:48.646407 15598 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:57:48.647595 15598 hybrid_clock.cc:648] HybridClock initialized: now 1786316268647552 us; error 30 us; skew 500 ppm
I20260809 22:57:48.647917 15598 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:57:48 dist-test-slave-ncp9 krb5kdc[15581](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316268, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:57:48.654479 15598 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 22:57:48.655869 15598 webserver.cc:460] Webserver started at http://127.0.0.1:38693/ using document root <none> and password file <none>
I20260809 22:57:48.656482 15598 fs_manager.cc:359] Metadata directory not provided
I20260809 22:57:48.656582 15598 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:57:48.656833 15598 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:57:48.659744 15598 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786315935617919-15814-0/minicluster-data/master-0/data/instance:
uuid: "8fa5bc13b96f4a9882cd7c96e145ce75"
format_stamp: "Formatted at 2026-08-09 22:57:48 on dist-test-slave-ncp9"
server_key: "66f93202a67f71ca939b08c850579b13"
server_key_iv: "1f7796d83f9ec6dd5b0a009ec626f636"
server_key_version: "encryptionkey@0"
I20260809 22:57:48.660548 15598 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786315935617919-15814-0/minicluster-data/master-0/wal/instance:
uuid: "8fa5bc13b96f4a9882cd7c96e145ce75"
format_stamp: "Formatted at 2026-08-09 22:57:48 on dist-test-slave-ncp9"
server_key: "66f93202a67f71ca939b08c850579b13"
server_key_iv: "1f7796d83f9ec6dd5b0a009ec626f636"
server_key_version: "encryptionkey@0"
I20260809 22:57:48.665410 15598 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.008s sys 0.000s
I20260809 22:57:48.668608 15625 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:57:48.669834 15598 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 22:57:48.669984 15598 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786315935617919-15814-0/minicluster-data/master-0/data,/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786315935617919-15814-0/minicluster-data/master-0/wal
uuid: "8fa5bc13b96f4a9882cd7c96e145ce75"
format_stamp: "Formatted at 2026-08-09 22:57:48 on dist-test-slave-ncp9"
server_key: "66f93202a67f71ca939b08c850579b13"
server_key_iv: "1f7796d83f9ec6dd5b0a009ec626f636"
server_key_version: "encryptionkey@0"
I20260809 22:57:48.670159 15598 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786315935617919-15814-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786315935617919-15814-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786315935617919-15814-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:57:48.688241 15598 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:57:48.692166 15598 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:57:48.692406 15598 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:57:48.709512 15598 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45595
I20260809 22:57:48.709545 15697 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45595 every 8 connection(s)
I20260809 22:57:48.710822 15598 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786315935617919-15814-0/minicluster-data/master-0/data/info.pb
I20260809 22:57:48.714392 15699 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:57:48.718904 15814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskItYPJR/build/asan/bin/kudu as pid 15598
I20260809 22:57:48.719095 15814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786315935617919-15814-0/minicluster-data/master-0/wal/instance
I20260809 22:57:48.719667 15814 external_mini_cluster.cc:1428] Setting key 4cd318288c555be0b9b122e27a7db139
I20260809 22:57:48.725405 15699 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8fa5bc13b96f4a9882cd7c96e145ce75: Bootstrap starting.
I20260809 22:57:48.728745 15699 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8fa5bc13b96f4a9882cd7c96e145ce75: Neither blocks nor log segments found. Creating new log.
I20260809 22:57:48.729923 15699 log.cc:824] T 00000000000000000000000000000000 P 8fa5bc13b96f4a9882cd7c96e145ce75: Log is configured to *not* fsync() on all Append() calls
I20260809 22:57:48.732506 15699 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8fa5bc13b96f4a9882cd7c96e145ce75: No bootstrap required, opened a new log
I20260809 22:57:48.732690 15706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:57:48.721393 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33861 (local address 127.0.0.1:45595)
0809 22:57:48.722210 (+ 817us) server_negotiation.cc:206] Beginning negotiation
0809 22:57:48.722221 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 22:57:48.722263 (+ 42us) server_negotiation.cc:407] Connection header received
0809 22:57:48.724020 (+ 1757us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:57:48.724042 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:57:48.724408 (+ 366us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:57:48.724720 (+ 312us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:57:48.725358 (+ 638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:57:48.726695 (+ 1337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:57:48.728477 (+ 1782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:57:48.729234 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:57:48.729358 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:57:48.730549 (+ 1191us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:57:48.730560 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:57:48.730573 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 22:57:48.731104 (+ 531us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:57:48.731176 (+ 72us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 22:57:48.731200 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:57:48.731298 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0809 22:57:48.731532 (+ 234us) server_negotiation.cc:299] Negotiation successful
0809 22:57:48.731784 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":590,"thread_start_us":357,"threads_started":1}
I20260809 22:57:48.736122 15814 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 22:57:48.739912 15699 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8fa5bc13b96f4a9882cd7c96e145ce75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8fa5bc13b96f4a9882cd7c96e145ce75" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45595 } }
I20260809 22:57:48.740315 15699 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8fa5bc13b96f4a9882cd7c96e145ce75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:57:48.740422 15699 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8fa5bc13b96f4a9882cd7c96e145ce75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8fa5bc13b96f4a9882cd7c96e145ce75, State: Initialized, Role: FOLLOWER
I20260809 22:57:48.741084 15699 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8fa5bc13b96f4a9882cd7c96e145ce75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8fa5bc13b96f4a9882cd7c96e145ce75" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45595 } }
I20260809 22:57:48.741360 15699 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8fa5bc13b96f4a9882cd7c96e145ce75 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 22:57:48.741509 15699 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8fa5bc13b96f4a9882cd7c96e145ce75 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 22:57:48.741720 15699 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8fa5bc13b96f4a9882cd7c96e145ce75 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:57:48.744284 15699 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8fa5bc13b96f4a9882cd7c96e145ce75 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8fa5bc13b96f4a9882cd7c96e145ce75" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45595 } }
I20260809 22:57:48.744853 15699 leader_election.cc:302] T 00000000000000000000000000000000 P 8fa5bc13b96f4a9882cd7c96e145ce75 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8fa5bc13b96f4a9882cd7c96e145ce75; no voters:
I20260809 22:57:48.745540 15699 leader_election.cc:288] T 00000000000000000000000000000000 P 8fa5bc13b96f4a9882cd7c96e145ce75 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 22:57:48.745656 15718 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8fa5bc13b96f4a9882cd7c96e145ce75 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 22:57:48.747309 15706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:57:48.740254 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46819 (local address 127.0.0.1:45595)
0809 22:57:48.740449 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0809 22:57:48.740455 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:57:48.740944 (+ 489us) server_negotiation.cc:407] Connection header received
0809 22:57:48.741099 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:57:48.741103 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:57:48.741160 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:57:48.741233 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:57:48.741852 (+ 619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:57:48.742923 (+ 1071us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:57:48.744253 (+ 1330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:57:48.744931 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:57:48.745051 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:57:48.746195 (+ 1144us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:57:48.746202 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:57:48.746208 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 22:57:48.746230 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:57:48.746280 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 22:57:48.746301 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:57:48.746382 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0809 22:57:48.746869 (+ 487us) server_negotiation.cc:299] Negotiation successful
0809 22:57:48.746962 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260809 22:57:48.747502 15699 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8fa5bc13b96f4a9882cd7c96e145ce75 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 22:57:48.747783 15718 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8fa5bc13b96f4a9882cd7c96e145ce75 [term 1 LEADER]: Becoming Leader. State: Replica: 8fa5bc13b96f4a9882cd7c96e145ce75, State: Running, Role: LEADER
I20260809 22:57:48.748371 15718 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8fa5bc13b96f4a9882cd7c96e145ce75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8fa5bc13b96f4a9882cd7c96e145ce75" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45595 } }
I20260809 22:57:48.751716 15720 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8fa5bc13b96f4a9882cd7c96e145ce75 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8fa5bc13b96f4a9882cd7c96e145ce75. Latest consensus state: current_term: 1 leader_uuid: "8fa5bc13b96f4a9882cd7c96e145ce75" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8fa5bc13b96f4a9882cd7c96e145ce75" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45595 } } }
I20260809 22:57:48.752094 15720 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8fa5bc13b96f4a9882cd7c96e145ce75 [sys.catalog]: This master's current role is: LEADER
I20260809 22:57:48.752673 15719 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8fa5bc13b96f4a9882cd7c96e145ce75 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8fa5bc13b96f4a9882cd7c96e145ce75" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8fa5bc13b96f4a9882cd7c96e145ce75" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45595 } } }
I20260809 22:57:48.752977 15719 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8fa5bc13b96f4a9882cd7c96e145ce75 [sys.catalog]: This master's current role is: LEADER
I20260809 22:57:48.755146 15727 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 22:57:48.759430 15727 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 22:57:48.767910 15727 catalog_manager.cc:1347] Generated new cluster ID: 5322e07ded9748ccafa947cb14aa5c96
I20260809 22:57:48.768002 15727 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 22:57:48.785089 15727 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 22:57:48.786275 15727 catalog_manager.cc:1504] Loading token signing keys...
I20260809 22:57:48.795311 15727 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8fa5bc13b96f4a9882cd7c96e145ce75: Generated new TSK 0
I20260809 22:57:48.796092 15727 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 22:57:48.810067 15814 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskItYPJR/build/asan/bin/kudu pid 15598
W20260809 22:57:48.811297 15691 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:33861
W20260809 22:57:48.811930 15814 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskItYPJR/build/asan/bin/kudu pid 15598: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 22:57:48.812084 15814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskItYPJR/build/asan/bin/kudu with pid 15598
2026-08-09T22:57:48Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (2060 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786315935617919-15814-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 22:57:48 dist-test-slave-ncp9 krb5kdc[15745](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 22:57:48 dist-test-slave-ncp9 krb5kdc[15745](info): set up 2 sockets
Aug 09 22:57:48 dist-test-slave-ncp9 krb5kdc[15745](info): commencing operation
krb5kdc: starting...
W20260809 22:57:50.031134 15814 mini_kdc.cc:121] Time spent starting KDC: real 1.175s user 0.001s sys 0.043s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786315935617919-15814-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786315935617919-15814-0/krb5kdc/test-user.keytab.
Aug 09 22:57:50 dist-test-slave-ncp9 krb5kdc[15745](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316270, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T22:57:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T22:57:50Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786315935617919-15814-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786315935617919-15814-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 22:57:50.319363 15814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskItYPJR/build/asan/bin/kudu
/tmp/dist-test-taskItYPJR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786315935617919-15814-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786315935617919-15814-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786315935617919-15814-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786315935617919-15814-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41089
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786315935617919-15814-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44675
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41089
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786315935617919-15814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786315935617919-15814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786315935617919-15814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:57:50.643574 15761 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:57:50.643857 15761 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:57:50.643985 15761 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:57:50.650980 15761 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 22:57:50.651087 15761 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:57:50.651162 15761 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:57:50.651247 15761 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 22:57:50.651299 15761 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 22:57:50.660811 15761 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44675
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786315935617919-15814-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786315935617919-15814-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41089
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786315935617919-15814-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41089
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786315935617919-15814-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786315935617919-15814-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:57:50.662061 15761 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:57:50.663349 15761 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:57:50.674391 15773 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:57:50.674436 15767 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:57:50.674531 15769 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:57:50.675099 15761 server_base.cc:1034] running on GCE node
I20260809 22:57:50.675756 15761 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:57:50.677053 15761 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:57:50.678265 15761 hybrid_clock.cc:648] HybridClock initialized: now 1786316270678221 us; error 36 us; skew 500 ppm
I20260809 22:57:50.678591 15761 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:57:50 dist-test-slave-ncp9 krb5kdc[15745](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316270, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:57:50.681823 15761 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 22:57:50.683212 15761 webserver.cc:460] Webserver started at http://127.0.0.1:34667/ using document root <none> and password file <none>
I20260809 22:57:50.683853 15761 fs_manager.cc:359] Metadata directory not provided
I20260809 22:57:50.683966 15761 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:57:50.684271 15761 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:57:50.687253 15761 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786315935617919-15814-0/minicluster-data/master-0/data/instance:
uuid: "f8d94a1ccd754ef6bf1db7f0def9519e"
format_stamp: "Formatted at 2026-08-09 22:57:50 on dist-test-slave-ncp9"
server_key: "0255a3f19d83c382d3e3a7088540c621"
server_key_iv: "4201ca2710dfed7862198751921e88a6"
server_key_version: "encryptionkey@0"
I20260809 22:57:50.687989 15761 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786315935617919-15814-0/minicluster-data/master-0/wal/instance:
uuid: "f8d94a1ccd754ef6bf1db7f0def9519e"
format_stamp: "Formatted at 2026-08-09 22:57:50 on dist-test-slave-ncp9"
server_key: "0255a3f19d83c382d3e3a7088540c621"
server_key_iv: "4201ca2710dfed7862198751921e88a6"
server_key_version: "encryptionkey@0"
I20260809 22:57:50.693045 15761 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.002s
I20260809 22:57:50.696372 15784 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:57:50.697678 15761 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 22:57:50.697829 15761 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786315935617919-15814-0/minicluster-data/master-0/data,/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786315935617919-15814-0/minicluster-data/master-0/wal
uuid: "f8d94a1ccd754ef6bf1db7f0def9519e"
format_stamp: "Formatted at 2026-08-09 22:57:50 on dist-test-slave-ncp9"
server_key: "0255a3f19d83c382d3e3a7088540c621"
server_key_iv: "4201ca2710dfed7862198751921e88a6"
server_key_version: "encryptionkey@0"
I20260809 22:57:50.698009 15761 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786315935617919-15814-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786315935617919-15814-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786315935617919-15814-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:57:50.703692 15761 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:57:50.707427 15761 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:57:50.707696 15761 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:57:50.724709 15761 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41089
I20260809 22:57:50.724753 15876 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41089 every 8 connection(s)
I20260809 22:57:50.726068 15761 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786315935617919-15814-0/minicluster-data/master-0/data/info.pb
I20260809 22:57:50.729647 15877 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:57:50.732738 15814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskItYPJR/build/asan/bin/kudu as pid 15761
I20260809 22:57:50.732954 15814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786315935617919-15814-0/minicluster-data/master-0/wal/instance
I20260809 22:57:50.733505 15814 external_mini_cluster.cc:1428] Setting key 287f89dbb7a9e9a8f9c98d22af6aec0b
I20260809 22:57:50.740972 15882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:57:50.735329 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39237 (local address 127.0.0.1:41089)
0809 22:57:50.736073 (+ 744us) server_negotiation.cc:206] Beginning negotiation
0809 22:57:50.736085 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 22:57:50.736125 (+ 40us) server_negotiation.cc:407] Connection header received
0809 22:57:50.737774 (+ 1649us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:57:50.737796 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:57:50.738127 (+ 331us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:57:50.738442 (+ 315us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:57:50.739474 (+ 1032us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:57:50.739480 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:57:50.739492 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 22:57:50.739507 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:57:50.739566 (+ 59us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 22:57:50.739602 (+ 36us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:57:50.739767 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0809 22:57:50.739891 (+ 124us) server_negotiation.cc:299] Negotiation successful
0809 22:57:50.740059 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":550,"thread_start_us":341,"threads_started":1}
I20260809 22:57:50.741556 15877 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f8d94a1ccd754ef6bf1db7f0def9519e: Bootstrap starting.
I20260809 22:57:50.744495 15814 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 22:57:50.744683 15877 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f8d94a1ccd754ef6bf1db7f0def9519e: Neither blocks nor log segments found. Creating new log.
I20260809 22:57:50.745908 15877 log.cc:824] T 00000000000000000000000000000000 P f8d94a1ccd754ef6bf1db7f0def9519e: Log is configured to *not* fsync() on all Append() calls
I20260809 22:57:50.749058 15877 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f8d94a1ccd754ef6bf1db7f0def9519e: No bootstrap required, opened a new log
I20260809 22:57:50.751780 15882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:57:50.749393 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47669 (local address 127.0.0.1:41089)
0809 22:57:50.749581 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0809 22:57:50.749587 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:57:50.750022 (+ 435us) server_negotiation.cc:407] Connection header received
0809 22:57:50.750176 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:57:50.750179 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:57:50.750222 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:57:50.750296 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:57:50.751207 (+ 911us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:57:50.751214 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:57:50.751217 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 22:57:50.751232 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:57:50.751267 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 22:57:50.751280 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:57:50.751366 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0809 22:57:50.751482 (+ 116us) server_negotiation.cc:299] Negotiation successful
0809 22:57:50.751530 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260809 22:57:50.758002 15877 raft_consensus.cc:348] T 00000000000000000000000000000000 P f8d94a1ccd754ef6bf1db7f0def9519e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f8d94a1ccd754ef6bf1db7f0def9519e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41089 } }
I20260809 22:57:50.758476 15877 raft_consensus.cc:374] T 00000000000000000000000000000000 P f8d94a1ccd754ef6bf1db7f0def9519e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:57:50.758582 15877 raft_consensus.cc:729] T 00000000000000000000000000000000 P f8d94a1ccd754ef6bf1db7f0def9519e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f8d94a1ccd754ef6bf1db7f0def9519e, State: Initialized, Role: FOLLOWER
I20260809 22:57:50.759191 15877 consensus_queue.cc:260] T 00000000000000000000000000000000 P f8d94a1ccd754ef6bf1db7f0def9519e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f8d94a1ccd754ef6bf1db7f0def9519e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41089 } }
I20260809 22:57:50.759413 15877 raft_consensus.cc:388] T 00000000000000000000000000000000 P f8d94a1ccd754ef6bf1db7f0def9519e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 22:57:50.759544 15877 raft_consensus.cc:482] T 00000000000000000000000000000000 P f8d94a1ccd754ef6bf1db7f0def9519e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 22:57:50.759684 15877 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f8d94a1ccd754ef6bf1db7f0def9519e [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:57:50.761811 15877 raft_consensus.cc:504] T 00000000000000000000000000000000 P f8d94a1ccd754ef6bf1db7f0def9519e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f8d94a1ccd754ef6bf1db7f0def9519e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41089 } }
I20260809 22:57:50.762305 15877 leader_election.cc:302] T 00000000000000000000000000000000 P f8d94a1ccd754ef6bf1db7f0def9519e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f8d94a1ccd754ef6bf1db7f0def9519e; no voters:
I20260809 22:57:50.762820 15877 leader_election.cc:288] T 00000000000000000000000000000000 P f8d94a1ccd754ef6bf1db7f0def9519e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 22:57:50.762938 15891 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f8d94a1ccd754ef6bf1db7f0def9519e [term 1 FOLLOWER]: Leader election won for term 1
I20260809 22:57:50.763578 15891 raft_consensus.cc:686] T 00000000000000000000000000000000 P f8d94a1ccd754ef6bf1db7f0def9519e [term 1 LEADER]: Becoming Leader. State: Replica: f8d94a1ccd754ef6bf1db7f0def9519e, State: Running, Role: LEADER
I20260809 22:57:50.764145 15891 consensus_queue.cc:237] T 00000000000000000000000000000000 P f8d94a1ccd754ef6bf1db7f0def9519e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f8d94a1ccd754ef6bf1db7f0def9519e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41089 } }
I20260809 22:57:50.764868 15877 sys_catalog.cc:564] T 00000000000000000000000000000000 P f8d94a1ccd754ef6bf1db7f0def9519e [sys.catalog]: configured and running, proceeding with master startup.
I20260809 22:57:50.766901 15895 sys_catalog.cc:455] T 00000000000000000000000000000000 P f8d94a1ccd754ef6bf1db7f0def9519e [sys.catalog]: SysCatalogTable state changed. Reason: New leader f8d94a1ccd754ef6bf1db7f0def9519e. Latest consensus state: current_term: 1 leader_uuid: "f8d94a1ccd754ef6bf1db7f0def9519e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f8d94a1ccd754ef6bf1db7f0def9519e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41089 } } }
I20260809 22:57:50.767373 15895 sys_catalog.cc:458] T 00000000000000000000000000000000 P f8d94a1ccd754ef6bf1db7f0def9519e [sys.catalog]: This master's current role is: LEADER
I20260809 22:57:50.767194 15892 sys_catalog.cc:455] T 00000000000000000000000000000000 P f8d94a1ccd754ef6bf1db7f0def9519e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f8d94a1ccd754ef6bf1db7f0def9519e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f8d94a1ccd754ef6bf1db7f0def9519e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41089 } } }
I20260809 22:57:50.767560 15892 sys_catalog.cc:458] T 00000000000000000000000000000000 P f8d94a1ccd754ef6bf1db7f0def9519e [sys.catalog]: This master's current role is: LEADER
I20260809 22:57:50.772149 15904 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 22:57:50.780618 15904 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 22:57:50.789052 15904 catalog_manager.cc:1347] Generated new cluster ID: 0b9d7533d29b4749b5f56d5722011df4
I20260809 22:57:50.789160 15904 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 22:57:50.808740 15904 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 22:57:50.809139 15904 catalog_manager.cc:1504] Loading token signing keys...
I20260809 22:57:50.815150 15904 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f8d94a1ccd754ef6bf1db7f0def9519e: Generated new TSK 0
I20260809 22:57:50.815850 15904 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 22:57:50.829165 15814 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskItYPJR/build/asan/bin/kudu pid 15761
W20260809 22:57:50.830456 15869 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:39237
W20260809 22:57:50.831148 15814 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskItYPJR/build/asan/bin/kudu pid 15761: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 22:57:50.831301 15814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskItYPJR/build/asan/bin/kudu with pid 15761
2026-08-09T22:57:50Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (2017 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786315935617919-15814-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 22:57:50 dist-test-slave-ncp9 krb5kdc[15929](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 22:57:50 dist-test-slave-ncp9 krb5kdc[15929](info): set up 2 sockets
Aug 09 22:57:50 dist-test-slave-ncp9 krb5kdc[15929](info): commencing operation
krb5kdc: starting...
W20260809 22:57:52.048105 15814 mini_kdc.cc:121] Time spent starting KDC: real 1.174s user 0.004s sys 0.043s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786315935617919-15814-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786315935617919-15814-0/krb5kdc/test-user.keytab.
Aug 09 22:57:52 dist-test-slave-ncp9 krb5kdc[15929](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316272, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T22:57:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T22:57:52Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786315935617919-15814-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786315935617919-15814-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 22:57:52.342023 15814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskItYPJR/build/asan/bin/kudu
/tmp/dist-test-taskItYPJR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786315935617919-15814-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786315935617919-15814-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786315935617919-15814-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786315935617919-15814-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36493
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786315935617919-15814-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38529
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36493
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786315935617919-15814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786315935617919-15814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786315935617919-15814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:57:52.666326 15947 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:57:52.666600 15947 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:57:52.666744 15947 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:57:52.673710 15947 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 22:57:52.673813 15947 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:57:52.673889 15947 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:57:52.673964 15947 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 22:57:52.674022 15947 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 22:57:52.683338 15947 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38529
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786315935617919-15814-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786315935617919-15814-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36493
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786315935617919-15814-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36493
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786315935617919-15814-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786315935617919-15814-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:57:52.684502 15947 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:57:52.685786 15947 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:57:52.696836 15955 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:57:52.696944 15954 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:57:52.696986 15960 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:57:52.698093 15947 server_base.cc:1034] running on GCE node
I20260809 22:57:52.698805 15947 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:57:52.700085 15947 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:57:52.701280 15947 hybrid_clock.cc:648] HybridClock initialized: now 1786316272701231 us; error 43 us; skew 500 ppm
I20260809 22:57:52.701594 15947 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:57:52 dist-test-slave-ncp9 krb5kdc[15929](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316272, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:57:52.704509 15947 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 22:57:52.705940 15947 webserver.cc:460] Webserver started at http://127.0.0.1:41403/ using document root <none> and password file <none>
I20260809 22:57:52.706580 15947 fs_manager.cc:359] Metadata directory not provided
I20260809 22:57:52.706702 15947 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:57:52.706986 15947 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:57:52.709923 15947 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786315935617919-15814-0/minicluster-data/master-0/data/instance:
uuid: "44057f6609ee4ee4bbb720ff8f99dfcb"
format_stamp: "Formatted at 2026-08-09 22:57:52 on dist-test-slave-ncp9"
server_key: "969adb4e432cee89d7fdccdba8fb3732"
server_key_iv: "650de9b37aa2fcf72bfe65d64462f564"
server_key_version: "encryptionkey@0"
I20260809 22:57:52.710685 15947 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786315935617919-15814-0/minicluster-data/master-0/wal/instance:
uuid: "44057f6609ee4ee4bbb720ff8f99dfcb"
format_stamp: "Formatted at 2026-08-09 22:57:52 on dist-test-slave-ncp9"
server_key: "969adb4e432cee89d7fdccdba8fb3732"
server_key_iv: "650de9b37aa2fcf72bfe65d64462f564"
server_key_version: "encryptionkey@0"
I20260809 22:57:52.715683 15947 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 22:57:52.718967 15968 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:57:52.720176 15947 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 22:57:52.720332 15947 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786315935617919-15814-0/minicluster-data/master-0/data,/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786315935617919-15814-0/minicluster-data/master-0/wal
uuid: "44057f6609ee4ee4bbb720ff8f99dfcb"
format_stamp: "Formatted at 2026-08-09 22:57:52 on dist-test-slave-ncp9"
server_key: "969adb4e432cee89d7fdccdba8fb3732"
server_key_iv: "650de9b37aa2fcf72bfe65d64462f564"
server_key_version: "encryptionkey@0"
I20260809 22:57:52.720502 15947 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786315935617919-15814-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786315935617919-15814-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786315935617919-15814-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:57:52.772396 15947 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:57:52.776574 15947 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:57:52.776832 15947 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:57:52.793797 15947 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36493
I20260809 22:57:52.793818 16046 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36493 every 8 connection(s)
I20260809 22:57:52.795032 15947 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786315935617919-15814-0/minicluster-data/master-0/data/info.pb
I20260809 22:57:52.796455 15814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskItYPJR/build/asan/bin/kudu as pid 15947
I20260809 22:57:52.796643 15814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786315935617919-15814-0/minicluster-data/master-0/wal/instance
I20260809 22:57:52.797245 15814 external_mini_cluster.cc:1428] Setting key bcb0f1646906c4a3fdd7e6f182d11d18
I20260809 22:57:52.799413 16049 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 09 22:57:52 dist-test-slave-ncp9 krb5kdc[15929](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316272, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260809 22:57:52.812129 16049 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 44057f6609ee4ee4bbb720ff8f99dfcb: Bootstrap starting.
I20260809 22:57:52.816192 16049 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 44057f6609ee4ee4bbb720ff8f99dfcb: Neither blocks nor log segments found. Creating new log.
I20260809 22:57:52.817466 16049 log.cc:824] T 00000000000000000000000000000000 P 44057f6609ee4ee4bbb720ff8f99dfcb: Log is configured to *not* fsync() on all Append() calls
I20260809 22:57:52.820523 16056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:57:52.799076 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.213:54201 (local address 127.0.0.1:36493)
0809 22:57:52.799835 (+ 759us) server_negotiation.cc:206] Beginning negotiation
0809 22:57:52.799848 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 22:57:52.799887 (+ 39us) server_negotiation.cc:407] Connection header received
0809 22:57:52.801502 (+ 1615us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:57:52.801524 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:57:52.801908 (+ 384us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:57:52.802235 (+ 327us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:57:52.804016 (+ 1781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:57:52.805369 (+ 1353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:57:52.807137 (+ 1768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:57:52.807789 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:57:52.807916 (+ 127us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:57:52.812529 (+ 4613us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:57:52.812541 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:57:52.812559 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:57:52.812606 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:57:52.816271 (+ 3665us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:57:52.817024 (+ 753us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:57:52.817033 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:57:52.817040 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:57:52.817267 (+ 227us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:57:52.817891 (+ 624us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:57:52.817898 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:57:52.817901 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:57:52.818627 (+ 726us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:57:52.818964 (+ 337us) server_negotiation.cc:1035] Waiting for connection context
0809 22:57:52.819286 (+ 322us) server_negotiation.cc:299] Negotiation successful
0809 22:57:52.819548 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":559,"thread_start_us":335,"threads_started":1}
I20260809 22:57:52.820791 16049 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 44057f6609ee4ee4bbb720ff8f99dfcb: No bootstrap required, opened a new log
I20260809 22:57:52.824096 15814 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 22:57:52.830152 16049 raft_consensus.cc:348] T 00000000000000000000000000000000 P 44057f6609ee4ee4bbb720ff8f99dfcb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44057f6609ee4ee4bbb720ff8f99dfcb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36493 } }
I20260809 22:57:52.830607 16049 raft_consensus.cc:374] T 00000000000000000000000000000000 P 44057f6609ee4ee4bbb720ff8f99dfcb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:57:52.830737 16049 raft_consensus.cc:729] T 00000000000000000000000000000000 P 44057f6609ee4ee4bbb720ff8f99dfcb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 44057f6609ee4ee4bbb720ff8f99dfcb, State: Initialized, Role: FOLLOWER
I20260809 22:57:52.831364 16049 consensus_queue.cc:260] T 00000000000000000000000000000000 P 44057f6609ee4ee4bbb720ff8f99dfcb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44057f6609ee4ee4bbb720ff8f99dfcb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36493 } }
I20260809 22:57:52.831646 16049 raft_consensus.cc:388] T 00000000000000000000000000000000 P 44057f6609ee4ee4bbb720ff8f99dfcb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 22:57:52.831791 16049 raft_consensus.cc:482] T 00000000000000000000000000000000 P 44057f6609ee4ee4bbb720ff8f99dfcb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 22:57:52.831995 16049 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 44057f6609ee4ee4bbb720ff8f99dfcb [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:57:52.834843 16049 raft_consensus.cc:504] T 00000000000000000000000000000000 P 44057f6609ee4ee4bbb720ff8f99dfcb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44057f6609ee4ee4bbb720ff8f99dfcb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36493 } }
I20260809 22:57:52.835395 16049 leader_election.cc:302] T 00000000000000000000000000000000 P 44057f6609ee4ee4bbb720ff8f99dfcb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 44057f6609ee4ee4bbb720ff8f99dfcb; no voters:
I20260809 22:57:52.836017 16049 leader_election.cc:288] T 00000000000000000000000000000000 P 44057f6609ee4ee4bbb720ff8f99dfcb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 22:57:52.836079 16068 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 44057f6609ee4ee4bbb720ff8f99dfcb [term 1 FOLLOWER]: Leader election won for term 1
I20260809 22:57:52.837168 16068 raft_consensus.cc:686] T 00000000000000000000000000000000 P 44057f6609ee4ee4bbb720ff8f99dfcb [term 1 LEADER]: Becoming Leader. State: Replica: 44057f6609ee4ee4bbb720ff8f99dfcb, State: Running, Role: LEADER
I20260809 22:57:52.837663 16068 consensus_queue.cc:237] T 00000000000000000000000000000000 P 44057f6609ee4ee4bbb720ff8f99dfcb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44057f6609ee4ee4bbb720ff8f99dfcb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36493 } }
I20260809 22:57:52.838215 16049 sys_catalog.cc:564] T 00000000000000000000000000000000 P 44057f6609ee4ee4bbb720ff8f99dfcb [sys.catalog]: configured and running, proceeding with master startup.
I20260809 22:57:52.840569 16069 sys_catalog.cc:455] T 00000000000000000000000000000000 P 44057f6609ee4ee4bbb720ff8f99dfcb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "44057f6609ee4ee4bbb720ff8f99dfcb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44057f6609ee4ee4bbb720ff8f99dfcb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36493 } } }
I20260809 22:57:52.841123 16069 sys_catalog.cc:458] T 00000000000000000000000000000000 P 44057f6609ee4ee4bbb720ff8f99dfcb [sys.catalog]: This master's current role is: LEADER
I20260809 22:57:52.842204 16071 sys_catalog.cc:455] T 00000000000000000000000000000000 P 44057f6609ee4ee4bbb720ff8f99dfcb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 44057f6609ee4ee4bbb720ff8f99dfcb. Latest consensus state: current_term: 1 leader_uuid: "44057f6609ee4ee4bbb720ff8f99dfcb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44057f6609ee4ee4bbb720ff8f99dfcb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36493 } } }
I20260809 22:57:52.842545 16071 sys_catalog.cc:458] T 00000000000000000000000000000000 P 44057f6609ee4ee4bbb720ff8f99dfcb [sys.catalog]: This master's current role is: LEADER
I20260809 22:57:52.846938 16056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:57:52.829117 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.213:58483 (local address 127.0.0.1:36493)
0809 22:57:52.829292 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0809 22:57:52.829298 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:57:52.829841 (+ 543us) server_negotiation.cc:407] Connection header received
0809 22:57:52.829996 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:57:52.829999 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:57:52.830055 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:57:52.830124 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:57:52.831917 (+ 1793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:57:52.832752 (+ 835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:57:52.834508 (+ 1756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:57:52.834911 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:57:52.835005 (+ 94us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:57:52.838057 (+ 3052us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:57:52.838063 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:57:52.838067 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:57:52.838101 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:57:52.842181 (+ 4080us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:57:52.843697 (+ 1516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:57:52.843703 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:57:52.843707 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:57:52.843885 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:57:52.844500 (+ 615us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:57:52.844506 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:57:52.844510 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:57:52.844944 (+ 434us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:57:52.846196 (+ 1252us) server_negotiation.cc:1035] Waiting for connection context
0809 22:57:52.846378 (+ 182us) server_negotiation.cc:299] Negotiation successful
0809 22:57:52.846568 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260809 22:57:52.849502 16076 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 22:57:52.854456 16076 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 22:57:52.863170 16076 catalog_manager.cc:1347] Generated new cluster ID: 84c29980a49944d1b33ba9a1a0c059e0
I20260809 22:57:52.863284 16076 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 22:57:52.889844 16076 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 22:57:52.891384 16076 catalog_manager.cc:1504] Loading token signing keys...
I20260809 22:57:52.902136 16076 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 44057f6609ee4ee4bbb720ff8f99dfcb: Generated new TSK 0
I20260809 22:57:52.902904 16076 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 22:57:52.908996 15814 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskItYPJR/build/asan/bin/kudu pid 15947
I20260809 22:57:52.910099 16040 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:57:53.001209 15814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskItYPJR/build/asan/bin/kudu with pid 15947
W20260809 22:57:53.026455 15925 connection.cc:535] client connection to 127.0.0.1:36493 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:36493): Connection reset by peer (error 104)
2026-08-09T22:57:53Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (2171 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786315935617919-15814-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 22:57:53 dist-test-slave-ncp9 krb5kdc[16100](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 22:57:53 dist-test-slave-ncp9 krb5kdc[16100](info): set up 2 sockets
Aug 09 22:57:53 dist-test-slave-ncp9 krb5kdc[16100](info): commencing operation
krb5kdc: starting...
W20260809 22:57:54.220517 15814 mini_kdc.cc:121] Time spent starting KDC: real 1.175s user 0.003s sys 0.040s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786315935617919-15814-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786315935617919-15814-0/krb5kdc/test-user.keytab.
Aug 09 22:57:54 dist-test-slave-ncp9 krb5kdc[16100](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316274, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T22:57:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T22:57:54Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786315935617919-15814-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786315935617919-15814-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 22:57:54.509428 15814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskItYPJR/build/asan/bin/kudu
/tmp/dist-test-taskItYPJR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786315935617919-15814-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786315935617919-15814-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786315935617919-15814-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786315935617919-15814-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43475
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786315935617919-15814-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43441
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43475
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786315935617919-15814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786315935617919-15814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786315935617919-15814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:57:54.844024 16122 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:57:54.844282 16122 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:57:54.844408 16122 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:57:54.851339 16122 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 22:57:54.851431 16122 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:57:54.851483 16122 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:57:54.851568 16122 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 22:57:54.851622 16122 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 22:57:54.861027 16122 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43441
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786315935617919-15814-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786315935617919-15814-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43475
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786315935617919-15814-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43475
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786315935617919-15814-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786315935617919-15814-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:57:54.862142 16122 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:57:54.863377 16122 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:57:54.874369 16134 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:57:54.874478 16130 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:57:54.874411 16131 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:57:54.875317 16122 server_base.cc:1034] running on GCE node
I20260809 22:57:54.876004 16122 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:57:54.877274 16122 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:57:54.878463 16122 hybrid_clock.cc:648] HybridClock initialized: now 1786316274878420 us; error 32 us; skew 500 ppm
I20260809 22:57:54.878772 16122 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:57:54 dist-test-slave-ncp9 krb5kdc[16100](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316274, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:57:54.881685 16122 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 22:57:54.883047 16122 webserver.cc:460] Webserver started at http://127.0.0.1:36873/ using document root <none> and password file <none>
I20260809 22:57:54.883642 16122 fs_manager.cc:359] Metadata directory not provided
I20260809 22:57:54.883765 16122 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:57:54.884044 16122 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:57:54.886857 16122 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786315935617919-15814-0/minicluster-data/master-0/data/instance:
uuid: "75e2ca012c9146fdb12a1c8fe1dae691"
format_stamp: "Formatted at 2026-08-09 22:57:54 on dist-test-slave-ncp9"
server_key: "3a4a52f116706632152fdb1db2f2193e"
server_key_iv: "c175091c8b0de825a0f338a425bc1132"
server_key_version: "encryptionkey@0"
I20260809 22:57:54.887573 16122 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786315935617919-15814-0/minicluster-data/master-0/wal/instance:
uuid: "75e2ca012c9146fdb12a1c8fe1dae691"
format_stamp: "Formatted at 2026-08-09 22:57:54 on dist-test-slave-ncp9"
server_key: "3a4a52f116706632152fdb1db2f2193e"
server_key_iv: "c175091c8b0de825a0f338a425bc1132"
server_key_version: "encryptionkey@0"
I20260809 22:57:54.892452 16122 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.002s
I20260809 22:57:54.895756 16144 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:57:54.897022 16122 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 22:57:54.897172 16122 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786315935617919-15814-0/minicluster-data/master-0/data,/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786315935617919-15814-0/minicluster-data/master-0/wal
uuid: "75e2ca012c9146fdb12a1c8fe1dae691"
format_stamp: "Formatted at 2026-08-09 22:57:54 on dist-test-slave-ncp9"
server_key: "3a4a52f116706632152fdb1db2f2193e"
server_key_iv: "c175091c8b0de825a0f338a425bc1132"
server_key_version: "encryptionkey@0"
I20260809 22:57:54.897338 16122 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786315935617919-15814-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786315935617919-15814-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786315935617919-15814-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:57:54.910815 16122 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:57:54.914484 16122 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:57:54.914740 16122 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:57:54.932029 16122 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43475
I20260809 22:57:54.932075 16225 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43475 every 8 connection(s)
I20260809 22:57:54.933333 16122 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786315935617919-15814-0/minicluster-data/master-0/data/info.pb
I20260809 22:57:54.936865 16226 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:57:54.938735 15814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskItYPJR/build/asan/bin/kudu as pid 16122
I20260809 22:57:54.938938 15814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786315935617919-15814-0/minicluster-data/master-0/wal/instance
I20260809 22:57:54.939525 15814 external_mini_cluster.cc:1428] Setting key 106078db3c5a4c183f05f13798d83314
I20260809 22:57:54.949288 16226 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 75e2ca012c9146fdb12a1c8fe1dae691: Bootstrap starting.
I20260809 22:57:54.951705 16230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:57:54.941353 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.213:52611 (local address 127.0.0.1:43475)
0809 22:57:54.942093 (+ 740us) server_negotiation.cc:206] Beginning negotiation
0809 22:57:54.942104 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 22:57:54.942143 (+ 39us) server_negotiation.cc:407] Connection header received
0809 22:57:54.943691 (+ 1548us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:57:54.943711 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:57:54.944136 (+ 425us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:57:54.944472 (+ 336us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:57:54.945006 (+ 534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:57:54.946291 (+ 1285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:57:54.947597 (+ 1306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:57:54.948224 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:57:54.948379 (+ 155us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:57:54.949628 (+ 1249us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:57:54.949635 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:57:54.949658 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 22:57:54.950142 (+ 484us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:57:54.950223 (+ 81us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 22:57:54.950247 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:57:54.950379 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0809 22:57:54.950700 (+ 321us) server_negotiation.cc:299] Negotiation successful
0809 22:57:54.950832 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":543,"thread_start_us":318,"threads_started":1}
I20260809 22:57:54.952500 16226 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 75e2ca012c9146fdb12a1c8fe1dae691: Neither blocks nor log segments found. Creating new log.
I20260809 22:57:54.953646 16226 log.cc:824] T 00000000000000000000000000000000 P 75e2ca012c9146fdb12a1c8fe1dae691: Log is configured to *not* fsync() on all Append() calls
I20260809 22:57:54.955384 15814 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 22:57:54.956351 16226 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 75e2ca012c9146fdb12a1c8fe1dae691: No bootstrap required, opened a new log
I20260809 22:57:54.963680 16226 raft_consensus.cc:348] T 00000000000000000000000000000000 P 75e2ca012c9146fdb12a1c8fe1dae691 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "75e2ca012c9146fdb12a1c8fe1dae691" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43475 } }
I20260809 22:57:54.964114 16226 raft_consensus.cc:374] T 00000000000000000000000000000000 P 75e2ca012c9146fdb12a1c8fe1dae691 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:57:54.964219 16226 raft_consensus.cc:729] T 00000000000000000000000000000000 P 75e2ca012c9146fdb12a1c8fe1dae691 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75e2ca012c9146fdb12a1c8fe1dae691, State: Initialized, Role: FOLLOWER
I20260809 22:57:54.964812 16226 consensus_queue.cc:260] T 00000000000000000000000000000000 P 75e2ca012c9146fdb12a1c8fe1dae691 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "75e2ca012c9146fdb12a1c8fe1dae691" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43475 } }
I20260809 22:57:54.965124 16230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:57:54.959429 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.213:51799 (local address 127.0.0.1:43475)
0809 22:57:54.959603 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0809 22:57:54.959608 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:57:54.959853 (+ 245us) server_negotiation.cc:407] Connection header received
0809 22:57:54.960017 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:57:54.960021 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:57:54.960082 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:57:54.960172 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:57:54.960626 (+ 454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:57:54.961429 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:57:54.962730 (+ 1301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:57:54.963149 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:57:54.963252 (+ 103us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:57:54.964329 (+ 1077us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:57:54.964335 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:57:54.964339 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 22:57:54.964357 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:57:54.964419 (+ 62us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 22:57:54.964435 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:57:54.964553 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0809 22:57:54.964738 (+ 185us) server_negotiation.cc:299] Negotiation successful
0809 22:57:54.964775 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260809 22:57:54.965132 16226 raft_consensus.cc:388] T 00000000000000000000000000000000 P 75e2ca012c9146fdb12a1c8fe1dae691 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 22:57:54.965327 16226 raft_consensus.cc:482] T 00000000000000000000000000000000 P 75e2ca012c9146fdb12a1c8fe1dae691 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 22:57:54.965502 16226 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 75e2ca012c9146fdb12a1c8fe1dae691 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:57:54.967623 16226 raft_consensus.cc:504] T 00000000000000000000000000000000 P 75e2ca012c9146fdb12a1c8fe1dae691 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "75e2ca012c9146fdb12a1c8fe1dae691" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43475 } }
I20260809 22:57:54.968099 16226 leader_election.cc:302] T 00000000000000000000000000000000 P 75e2ca012c9146fdb12a1c8fe1dae691 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 75e2ca012c9146fdb12a1c8fe1dae691; no voters:
I20260809 22:57:54.968608 16226 leader_election.cc:288] T 00000000000000000000000000000000 P 75e2ca012c9146fdb12a1c8fe1dae691 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 22:57:54.968719 16240 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 75e2ca012c9146fdb12a1c8fe1dae691 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 22:57:54.969403 16240 raft_consensus.cc:686] T 00000000000000000000000000000000 P 75e2ca012c9146fdb12a1c8fe1dae691 [term 1 LEADER]: Becoming Leader. State: Replica: 75e2ca012c9146fdb12a1c8fe1dae691, State: Running, Role: LEADER
I20260809 22:57:54.969960 16240 consensus_queue.cc:237] T 00000000000000000000000000000000 P 75e2ca012c9146fdb12a1c8fe1dae691 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "75e2ca012c9146fdb12a1c8fe1dae691" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43475 } }
I20260809 22:57:54.970628 16226 sys_catalog.cc:564] T 00000000000000000000000000000000 P 75e2ca012c9146fdb12a1c8fe1dae691 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 22:57:54.972863 16243 sys_catalog.cc:455] T 00000000000000000000000000000000 P 75e2ca012c9146fdb12a1c8fe1dae691 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "75e2ca012c9146fdb12a1c8fe1dae691" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "75e2ca012c9146fdb12a1c8fe1dae691" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43475 } } }
I20260809 22:57:54.973505 16243 sys_catalog.cc:458] T 00000000000000000000000000000000 P 75e2ca012c9146fdb12a1c8fe1dae691 [sys.catalog]: This master's current role is: LEADER
I20260809 22:57:54.974946 16261 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 22:57:54.974833 16244 sys_catalog.cc:455] T 00000000000000000000000000000000 P 75e2ca012c9146fdb12a1c8fe1dae691 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 75e2ca012c9146fdb12a1c8fe1dae691. Latest consensus state: current_term: 1 leader_uuid: "75e2ca012c9146fdb12a1c8fe1dae691" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "75e2ca012c9146fdb12a1c8fe1dae691" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43475 } } }
I20260809 22:57:54.975148 16244 sys_catalog.cc:458] T 00000000000000000000000000000000 P 75e2ca012c9146fdb12a1c8fe1dae691 [sys.catalog]: This master's current role is: LEADER
I20260809 22:57:54.978981 16261 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 22:57:54.988601 16261 catalog_manager.cc:1347] Generated new cluster ID: e3f3b573fef946d6be580abd4d7dc9ab
I20260809 22:57:54.988703 16261 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 22:57:55.006428 16261 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 22:57:55.007622 16261 catalog_manager.cc:1504] Loading token signing keys...
I20260809 22:57:55.016201 16261 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 75e2ca012c9146fdb12a1c8fe1dae691: Generated new TSK 0
I20260809 22:57:55.017021 16261 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 22:57:55.023783 15814 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskItYPJR/build/asan/bin/kudu pid 16122
W20260809 22:57:55.025125 16213 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.213:52611
W20260809 22:57:55.025805 15814 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskItYPJR/build/asan/bin/kudu pid 16122: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 22:57:55.025929 15814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskItYPJR/build/asan/bin/kudu with pid 16122
W20260809 22:57:55.050321 16097 connection.cc:535] client connection to 127.0.0.1:43475 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:43475): Connection reset by peer (error 104)
2026-08-09T22:57:55Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (2024 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786315935617919-15814-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 22:57:55 dist-test-slave-ncp9 krb5kdc[16274](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 22:57:55 dist-test-slave-ncp9 krb5kdc[16274](info): set up 2 sockets
Aug 09 22:57:55 dist-test-slave-ncp9 krb5kdc[16274](info): commencing operation
krb5kdc: starting...
W20260809 22:57:56.255353 15814 mini_kdc.cc:121] Time spent starting KDC: real 1.186s user 0.000s sys 0.043s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786315935617919-15814-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786315935617919-15814-0/krb5kdc/test-user.keytab.
Aug 09 22:57:56 dist-test-slave-ncp9 krb5kdc[16274](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316276, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T22:57:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T22:57:56Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786315935617919-15814-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786315935617919-15814-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 22:57:56.547276 15814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskItYPJR/build/asan/bin/kudu
/tmp/dist-test-taskItYPJR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786315935617919-15814-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786315935617919-15814-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786315935617919-15814-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786315935617919-15814-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46215
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786315935617919-15814-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39061
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46215
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786315935617919-15814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786315935617919-15814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786315935617919-15814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:57:56.877130 16291 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:57:56.877404 16291 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:57:56.877511 16291 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:57:56.884407 16291 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 22:57:56.884506 16291 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:57:56.884560 16291 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:57:56.884619 16291 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 22:57:56.884662 16291 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 22:57:56.894126 16291 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39061
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786315935617919-15814-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786315935617919-15814-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46215
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786315935617919-15814-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46215
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786315935617919-15814-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786315935617919-15814-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:57:56.895270 16291 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:57:56.896512 16291 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:57:56.907747 16301 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:57:56.907886 16300 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:57:56.908262 16304 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:57:56.908814 16291 server_base.cc:1034] running on GCE node
I20260809 22:57:56.909569 16291 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:57:56.910889 16291 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:57:56.912101 16291 hybrid_clock.cc:648] HybridClock initialized: now 1786316276912043 us; error 41 us; skew 500 ppm
I20260809 22:57:56.912426 16291 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:57:56 dist-test-slave-ncp9 krb5kdc[16274](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316276, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:57:56.915445 16291 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 22:57:56.916803 16291 webserver.cc:460] Webserver started at http://127.0.0.1:33047/ using document root <none> and password file <none>
I20260809 22:57:56.917469 16291 fs_manager.cc:359] Metadata directory not provided
I20260809 22:57:56.917590 16291 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:57:56.917873 16291 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:57:56.920761 16291 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786315935617919-15814-0/minicluster-data/master-0/data/instance:
uuid: "0700fde9870143eda8270e87101f4383"
format_stamp: "Formatted at 2026-08-09 22:57:56 on dist-test-slave-ncp9"
server_key: "8dd573a21f12d95b75651ba43e795f30"
server_key_iv: "0d58332680f2bffe6e28fbd5e5ff41e2"
server_key_version: "encryptionkey@0"
I20260809 22:57:56.921551 16291 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786315935617919-15814-0/minicluster-data/master-0/wal/instance:
uuid: "0700fde9870143eda8270e87101f4383"
format_stamp: "Formatted at 2026-08-09 22:57:56 on dist-test-slave-ncp9"
server_key: "8dd573a21f12d95b75651ba43e795f30"
server_key_iv: "0d58332680f2bffe6e28fbd5e5ff41e2"
server_key_version: "encryptionkey@0"
I20260809 22:57:56.926486 16291 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260809 22:57:56.929740 16315 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:57:56.930940 16291 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 22:57:56.931102 16291 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786315935617919-15814-0/minicluster-data/master-0/data,/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786315935617919-15814-0/minicluster-data/master-0/wal
uuid: "0700fde9870143eda8270e87101f4383"
format_stamp: "Formatted at 2026-08-09 22:57:56 on dist-test-slave-ncp9"
server_key: "8dd573a21f12d95b75651ba43e795f30"
server_key_iv: "0d58332680f2bffe6e28fbd5e5ff41e2"
server_key_version: "encryptionkey@0"
I20260809 22:57:56.931280 16291 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786315935617919-15814-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786315935617919-15814-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786315935617919-15814-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:57:56.936560 16291 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:57:56.940510 16291 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:57:56.940779 16291 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:57:56.957825 16291 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46215
I20260809 22:57:56.957870 16392 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46215 every 8 connection(s)
I20260809 22:57:56.959115 16291 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786315935617919-15814-0/minicluster-data/master-0/data/info.pb
I20260809 22:57:56.962666 16396 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:57:56.963642 15814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskItYPJR/build/asan/bin/kudu as pid 16291
I20260809 22:57:56.963835 15814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786315935617919-15814-0/minicluster-data/master-0/wal/instance
I20260809 22:57:56.964423 15814 external_mini_cluster.cc:1428] Setting key a7ff59883538f3715f4f318e1453751a
I20260809 22:57:56.971900 16400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:57:56.966168 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.213:39451 (local address 127.0.0.1:46215)
0809 22:57:56.966898 (+ 730us) server_negotiation.cc:206] Beginning negotiation
0809 22:57:56.966912 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0809 22:57:56.966955 (+ 43us) server_negotiation.cc:407] Connection header received
0809 22:57:56.968603 (+ 1648us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:57:56.968623 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:57:56.969029 (+ 406us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:57:56.969339 (+ 310us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:57:56.970356 (+ 1017us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:57:56.970363 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:57:56.970377 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 22:57:56.970391 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:57:56.970449 (+ 58us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 22:57:56.970472 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:57:56.970627 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0809 22:57:56.970744 (+ 117us) server_negotiation.cc:299] Negotiation successful
0809 22:57:56.970909 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":513,"thread_start_us":326,"threads_started":1}
I20260809 22:57:56.975435 16396 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0700fde9870143eda8270e87101f4383: Bootstrap starting.
I20260809 22:57:56.975646 15814 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 22:57:56.979153 16396 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0700fde9870143eda8270e87101f4383: Neither blocks nor log segments found. Creating new log.
I20260809 22:57:56.980434 16396 log.cc:824] T 00000000000000000000000000000000 P 0700fde9870143eda8270e87101f4383: Log is configured to *not* fsync() on all Append() calls
I20260809 22:57:56.983275 16400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:57:56.980808 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.213:59345 (local address 127.0.0.1:46215)
0809 22:57:56.981007 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0809 22:57:56.981013 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:57:56.981493 (+ 480us) server_negotiation.cc:407] Connection header received
0809 22:57:56.981699 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:57:56.981704 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:57:56.981768 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:57:56.981876 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:57:56.982731 (+ 855us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:57:56.982735 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:57:56.982739 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 22:57:56.982753 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:57:56.982788 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 22:57:56.982802 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:57:56.982880 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0809 22:57:56.982991 (+ 111us) server_negotiation.cc:299] Negotiation successful
0809 22:57:56.983035 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260809 22:57:56.984129 16396 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0700fde9870143eda8270e87101f4383: No bootstrap required, opened a new log
I20260809 22:57:56.990936 16396 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0700fde9870143eda8270e87101f4383 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0700fde9870143eda8270e87101f4383" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46215 } }
I20260809 22:57:56.991262 16396 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0700fde9870143eda8270e87101f4383 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:57:56.991348 16396 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0700fde9870143eda8270e87101f4383 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0700fde9870143eda8270e87101f4383, State: Initialized, Role: FOLLOWER
I20260809 22:57:56.991873 16396 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0700fde9870143eda8270e87101f4383 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0700fde9870143eda8270e87101f4383" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46215 } }
I20260809 22:57:56.992107 16396 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0700fde9870143eda8270e87101f4383 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 22:57:56.992246 16396 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0700fde9870143eda8270e87101f4383 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 22:57:56.992394 16396 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0700fde9870143eda8270e87101f4383 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:57:56.994532 16396 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0700fde9870143eda8270e87101f4383 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0700fde9870143eda8270e87101f4383" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46215 } }
I20260809 22:57:56.994988 16396 leader_election.cc:302] T 00000000000000000000000000000000 P 0700fde9870143eda8270e87101f4383 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0700fde9870143eda8270e87101f4383; no voters:
I20260809 22:57:56.995478 16396 leader_election.cc:288] T 00000000000000000000000000000000 P 0700fde9870143eda8270e87101f4383 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 22:57:56.995608 16411 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0700fde9870143eda8270e87101f4383 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 22:57:56.996199 16411 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0700fde9870143eda8270e87101f4383 [term 1 LEADER]: Becoming Leader. State: Replica: 0700fde9870143eda8270e87101f4383, State: Running, Role: LEADER
I20260809 22:57:56.996747 16411 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0700fde9870143eda8270e87101f4383 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0700fde9870143eda8270e87101f4383" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46215 } }
I20260809 22:57:56.997526 16396 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0700fde9870143eda8270e87101f4383 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 22:57:56.999351 16412 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0700fde9870143eda8270e87101f4383 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0700fde9870143eda8270e87101f4383" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0700fde9870143eda8270e87101f4383" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46215 } } }
I20260809 22:57:56.999774 16412 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0700fde9870143eda8270e87101f4383 [sys.catalog]: This master's current role is: LEADER
I20260809 22:57:56.999668 16413 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0700fde9870143eda8270e87101f4383 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0700fde9870143eda8270e87101f4383. Latest consensus state: current_term: 1 leader_uuid: "0700fde9870143eda8270e87101f4383" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0700fde9870143eda8270e87101f4383" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46215 } } }
I20260809 22:57:56.999923 16413 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0700fde9870143eda8270e87101f4383 [sys.catalog]: This master's current role is: LEADER
I20260809 22:57:57.001814 16423 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 22:57:57.006546 16423 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 22:57:57.014745 16423 catalog_manager.cc:1347] Generated new cluster ID: d25674d8dc3e4c4dbf04fb235aeaa6ce
I20260809 22:57:57.014839 16423 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 22:57:57.035027 16423 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 22:57:57.035481 16423 catalog_manager.cc:1504] Loading token signing keys...
I20260809 22:57:57.049293 16423 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0700fde9870143eda8270e87101f4383: Generated new TSK 0
I20260809 22:57:57.050457 16423 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 22:57:57.062312 15814 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskItYPJR/build/asan/bin/kudu pid 16291
W20260809 22:57:57.064024 16387 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.213:39451
W20260809 22:57:57.065218 15814 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskItYPJR/build/asan/bin/kudu pid 16291: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 22:57:57.065434 15814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskItYPJR/build/asan/bin/kudu with pid 16291
2026-08-09T22:57:57Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (2055 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786315935617919-15814-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 22:57:57 dist-test-slave-ncp9 krb5kdc[16437](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 22:57:57 dist-test-slave-ncp9 krb5kdc[16437](info): set up 2 sockets
Aug 09 22:57:57 dist-test-slave-ncp9 krb5kdc[16437](info): commencing operation
krb5kdc: starting...
W20260809 22:57:59.019366 15814 mini_kdc.cc:121] Time spent starting KDC: real 1.890s user 0.000s sys 0.077s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786315935617919-15814-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786315935617919-15814-0/krb5kdc/test-user.keytab.
Aug 09 22:57:59 dist-test-slave-ncp9 krb5kdc[16437](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316279, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T22:57:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T22:57:59Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786315935617919-15814-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786315935617919-15814-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 22:57:59.311602 15814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskItYPJR/build/asan/bin/kudu
/tmp/dist-test-taskItYPJR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786315935617919-15814-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786315935617919-15814-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786315935617919-15814-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786315935617919-15814-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40051
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786315935617919-15814-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44085
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40051
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786315935617919-15814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786315935617919-15814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786315935617919-15814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:57:59.650991 16458 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:57:59.651275 16458 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:57:59.651407 16458 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:57:59.658427 16458 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 22:57:59.658527 16458 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:57:59.658604 16458 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:57:59.658681 16458 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 22:57:59.658738 16458 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 22:57:59.668373 16458 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44085
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786315935617919-15814-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786315935617919-15814-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40051
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786315935617919-15814-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40051
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786315935617919-15814-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786315935617919-15814-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:57:59.669658 16458 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:57:59.670912 16458 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:57:59.682219 16472 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:57:59.682516 16469 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:57:59.683025 16468 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:57:59.683230 16458 server_base.cc:1034] running on GCE node
I20260809 22:57:59.683966 16458 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:57:59.685312 16458 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:57:59.686506 16458 hybrid_clock.cc:648] HybridClock initialized: now 1786316279686465 us; error 29 us; skew 500 ppm
I20260809 22:57:59.686836 16458 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:57:59 dist-test-slave-ncp9 krb5kdc[16437](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316279, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:57:59.693248 16458 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 22:57:59.694674 16458 webserver.cc:460] Webserver started at http://127.0.0.1:42979/ using document root <none> and password file <none>
I20260809 22:57:59.695338 16458 fs_manager.cc:359] Metadata directory not provided
I20260809 22:57:59.695463 16458 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:57:59.695781 16458 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:57:59.698778 16458 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786315935617919-15814-0/minicluster-data/master-0/data/instance:
uuid: "25ebc48822a948f5b9d5004995ef46ef"
format_stamp: "Formatted at 2026-08-09 22:57:59 on dist-test-slave-ncp9"
server_key: "8918ba47257910b7313a122eea5e02b2"
server_key_iv: "e9a51503455f78e3864663c048d1664c"
server_key_version: "encryptionkey@0"
I20260809 22:57:59.699545 16458 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786315935617919-15814-0/minicluster-data/master-0/wal/instance:
uuid: "25ebc48822a948f5b9d5004995ef46ef"
format_stamp: "Formatted at 2026-08-09 22:57:59 on dist-test-slave-ncp9"
server_key: "8918ba47257910b7313a122eea5e02b2"
server_key_iv: "e9a51503455f78e3864663c048d1664c"
server_key_version: "encryptionkey@0"
I20260809 22:57:59.704578 16458 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260809 22:57:59.707999 16485 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:57:59.709323 16458 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260809 22:57:59.709487 16458 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786315935617919-15814-0/minicluster-data/master-0/data,/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786315935617919-15814-0/minicluster-data/master-0/wal
uuid: "25ebc48822a948f5b9d5004995ef46ef"
format_stamp: "Formatted at 2026-08-09 22:57:59 on dist-test-slave-ncp9"
server_key: "8918ba47257910b7313a122eea5e02b2"
server_key_iv: "e9a51503455f78e3864663c048d1664c"
server_key_version: "encryptionkey@0"
I20260809 22:57:59.709667 16458 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786315935617919-15814-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786315935617919-15814-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786315935617919-15814-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:57:59.729163 16458 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:57:59.732863 16458 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:57:59.733177 16458 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:57:59.750383 16458 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40051
I20260809 22:57:59.750415 16562 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40051 every 8 connection(s)
I20260809 22:57:59.751701 16458 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786315935617919-15814-0/minicluster-data/master-0/data/info.pb
I20260809 22:57:59.755316 16564 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:57:59.758549 15814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskItYPJR/build/asan/bin/kudu as pid 16458
I20260809 22:57:59.758749 15814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786315935617919-15814-0/minicluster-data/master-0/wal/instance
I20260809 22:57:59.759299 15814 external_mini_cluster.cc:1428] Setting key a332906d0f533a9d1b103804c0742898
I20260809 22:57:59.766808 16564 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 25ebc48822a948f5b9d5004995ef46ef: Bootstrap starting.
I20260809 22:57:59.769695 16564 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 25ebc48822a948f5b9d5004995ef46ef: Neither blocks nor log segments found. Creating new log.
I20260809 22:57:59.770743 16564 log.cc:824] T 00000000000000000000000000000000 P 25ebc48822a948f5b9d5004995ef46ef: Log is configured to *not* fsync() on all Append() calls
Aug 09 22:57:59 dist-test-slave-ncp9 krb5kdc[16437](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316279, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260809 22:57:59.773595 16564 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 25ebc48822a948f5b9d5004995ef46ef: No bootstrap required, opened a new log
I20260809 22:57:59.780970 16569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:57:59.761251 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.213:56275 (local address 127.0.0.1:40051)
0809 22:57:59.761932 (+ 681us) server_negotiation.cc:206] Beginning negotiation
0809 22:57:59.761944 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 22:57:59.761984 (+ 40us) server_negotiation.cc:407] Connection header received
0809 22:57:59.763658 (+ 1674us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:57:59.763683 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:57:59.764088 (+ 405us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:57:59.764456 (+ 368us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:57:59.765942 (+ 1486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:57:59.767275 (+ 1333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:57:59.768538 (+ 1263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:57:59.769238 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:57:59.769417 (+ 179us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:57:59.773823 (+ 4406us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:57:59.773831 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:57:59.773845 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:57:59.773878 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:57:59.776925 (+ 3047us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:57:59.777708 (+ 783us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:57:59.777714 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:57:59.777720 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:57:59.777888 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:57:59.778360 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:57:59.778364 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:57:59.778366 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:57:59.778885 (+ 519us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:57:59.779208 (+ 323us) server_negotiation.cc:1035] Waiting for connection context
0809 22:57:59.779738 (+ 530us) server_negotiation.cc:299] Negotiation successful
0809 22:57:59.780012 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":480,"thread_start_us":279,"threads_started":1}
I20260809 22:57:59.781373 16564 raft_consensus.cc:348] T 00000000000000000000000000000000 P 25ebc48822a948f5b9d5004995ef46ef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25ebc48822a948f5b9d5004995ef46ef" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40051 } }
I20260809 22:57:59.781713 16564 raft_consensus.cc:374] T 00000000000000000000000000000000 P 25ebc48822a948f5b9d5004995ef46ef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:57:59.781829 16564 raft_consensus.cc:729] T 00000000000000000000000000000000 P 25ebc48822a948f5b9d5004995ef46ef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 25ebc48822a948f5b9d5004995ef46ef, State: Initialized, Role: FOLLOWER
I20260809 22:57:59.782395 16564 consensus_queue.cc:260] T 00000000000000000000000000000000 P 25ebc48822a948f5b9d5004995ef46ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25ebc48822a948f5b9d5004995ef46ef" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40051 } }
I20260809 22:57:59.782617 16564 raft_consensus.cc:388] T 00000000000000000000000000000000 P 25ebc48822a948f5b9d5004995ef46ef [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 22:57:59.782753 16564 raft_consensus.cc:482] T 00000000000000000000000000000000 P 25ebc48822a948f5b9d5004995ef46ef [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 22:57:59.782918 16564 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 25ebc48822a948f5b9d5004995ef46ef [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:57:59.784588 15814 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 22:57:59.785362 16564 raft_consensus.cc:504] T 00000000000000000000000000000000 P 25ebc48822a948f5b9d5004995ef46ef [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25ebc48822a948f5b9d5004995ef46ef" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40051 } }
I20260809 22:57:59.785810 16564 leader_election.cc:302] T 00000000000000000000000000000000 P 25ebc48822a948f5b9d5004995ef46ef [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 25ebc48822a948f5b9d5004995ef46ef; no voters:
I20260809 22:57:59.786360 16564 leader_election.cc:288] T 00000000000000000000000000000000 P 25ebc48822a948f5b9d5004995ef46ef [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 22:57:59.786512 16574 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 25ebc48822a948f5b9d5004995ef46ef [term 1 FOLLOWER]: Leader election won for term 1
I20260809 22:57:59.788067 16564 sys_catalog.cc:564] T 00000000000000000000000000000000 P 25ebc48822a948f5b9d5004995ef46ef [sys.catalog]: configured and running, proceeding with master startup.
I20260809 22:57:59.793514 16574 raft_consensus.cc:686] T 00000000000000000000000000000000 P 25ebc48822a948f5b9d5004995ef46ef [term 1 LEADER]: Becoming Leader. State: Replica: 25ebc48822a948f5b9d5004995ef46ef, State: Running, Role: LEADER
I20260809 22:57:59.794193 16574 consensus_queue.cc:237] T 00000000000000000000000000000000 P 25ebc48822a948f5b9d5004995ef46ef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25ebc48822a948f5b9d5004995ef46ef" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40051 } }
I20260809 22:57:59.797297 16575 sys_catalog.cc:455] T 00000000000000000000000000000000 P 25ebc48822a948f5b9d5004995ef46ef [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "25ebc48822a948f5b9d5004995ef46ef" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25ebc48822a948f5b9d5004995ef46ef" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40051 } } }
I20260809 22:57:59.797356 16578 sys_catalog.cc:455] T 00000000000000000000000000000000 P 25ebc48822a948f5b9d5004995ef46ef [sys.catalog]: SysCatalogTable state changed. Reason: New leader 25ebc48822a948f5b9d5004995ef46ef. Latest consensus state: current_term: 1 leader_uuid: "25ebc48822a948f5b9d5004995ef46ef" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25ebc48822a948f5b9d5004995ef46ef" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40051 } } }
I20260809 22:57:59.797742 16575 sys_catalog.cc:458] T 00000000000000000000000000000000 P 25ebc48822a948f5b9d5004995ef46ef [sys.catalog]: This master's current role is: LEADER
I20260809 22:57:59.797742 16578 sys_catalog.cc:458] T 00000000000000000000000000000000 P 25ebc48822a948f5b9d5004995ef46ef [sys.catalog]: This master's current role is: LEADER
I20260809 22:57:59.813915 16590 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 22:57:59.818797 16590 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 22:57:59.829103 16590 catalog_manager.cc:1347] Generated new cluster ID: ce706b1b15584915be6603e101d1f86c
I20260809 22:57:59.829102 16569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:57:59.799751 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.213:53705 (local address 127.0.0.1:40051)
0809 22:57:59.799915 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0809 22:57:59.799921 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:57:59.813454 (+ 13533us) server_negotiation.cc:407] Connection header received
0809 22:57:59.813789 (+ 335us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:57:59.813795 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:57:59.813883 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:57:59.813991 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:57:59.815487 (+ 1496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:57:59.816587 (+ 1100us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:57:59.818339 (+ 1752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:57:59.818994 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:57:59.819116 (+ 122us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:57:59.822322 (+ 3206us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:57:59.822329 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:57:59.822334 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:57:59.822372 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:57:59.826099 (+ 3727us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:57:59.826938 (+ 839us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:57:59.826942 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:57:59.826944 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:57:59.827062 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:57:59.827508 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:57:59.827515 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:57:59.827517 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:57:59.827884 (+ 367us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:57:59.828142 (+ 258us) server_negotiation.cc:1035] Waiting for connection context
0809 22:57:59.828561 (+ 419us) server_negotiation.cc:299] Negotiation successful
0809 22:57:59.828737 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260809 22:57:59.829241 16590 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 22:57:59.881403 16590 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 22:57:59.882719 16590 catalog_manager.cc:1504] Loading token signing keys...
I20260809 22:57:59.888224 16590 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 25ebc48822a948f5b9d5004995ef46ef: Generated new TSK 0
I20260809 22:57:59.889145 16590 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 22:57:59.897830 15814 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskItYPJR/build/asan/bin/kudu pid 16458
I20260809 22:57:59.898867 16555 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:57:59.990571 15814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskItYPJR/build/asan/bin/kudu with pid 16458
W20260809 22:58:00.013826 16434 connection.cc:535] client connection to 127.0.0.1:40051 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:40051): Connection reset by peer (error 104)
2026-08-09T22:58:00Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (2907 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786315935617919-15814-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 22:58:00 dist-test-slave-ncp9 krb5kdc[16612](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 22:58:00 dist-test-slave-ncp9 krb5kdc[16612](info): set up 2 sockets
Aug 09 22:58:00 dist-test-slave-ncp9 krb5kdc[16612](info): commencing operation
krb5kdc: starting...
W20260809 22:58:01.238322 15814 mini_kdc.cc:121] Time spent starting KDC: real 1.206s user 0.001s sys 0.048s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786315935617919-15814-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786315935617919-15814-0/krb5kdc/test-user.keytab.
Aug 09 22:58:01 dist-test-slave-ncp9 krb5kdc[16612](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316281, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T22:58:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T22:58:01Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786315935617919-15814-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786315935617919-15814-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 22:58:01.520304 15814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskItYPJR/build/asan/bin/kudu
/tmp/dist-test-taskItYPJR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786315935617919-15814-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786315935617919-15814-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786315935617919-15814-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786315935617919-15814-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42383
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786315935617919-15814-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34597
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42383
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786315935617919-15814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786315935617919-15814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786315935617919-15814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:58:01.858875 16632 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:58:01.859158 16632 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:58:01.859301 16632 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:58:01.866426 16632 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 22:58:01.866549 16632 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:58:01.866624 16632 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:58:01.866706 16632 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 22:58:01.866766 16632 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 22:58:01.876210 16632 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34597
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786315935617919-15814-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786315935617919-15814-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42383
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786315935617919-15814-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42383
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786315935617919-15814-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786315935617919-15814-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:58:01.877444 16632 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:58:01.878752 16632 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:58:01.889890 16645 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:58:01.889972 16642 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:58:01.889942 16643 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:58:01.890866 16632 server_base.cc:1034] running on GCE node
I20260809 22:58:01.891578 16632 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:58:01.892836 16632 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:58:01.894057 16632 hybrid_clock.cc:648] HybridClock initialized: now 1786316281894013 us; error 34 us; skew 500 ppm
I20260809 22:58:01.894364 16632 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:58:01 dist-test-slave-ncp9 krb5kdc[16612](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316281, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:58:01.897503 16632 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 22:58:01.898877 16632 webserver.cc:460] Webserver started at http://127.0.0.1:38727/ using document root <none> and password file <none>
I20260809 22:58:01.899540 16632 fs_manager.cc:359] Metadata directory not provided
I20260809 22:58:01.899659 16632 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:58:01.899935 16632 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:58:01.902788 16632 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786315935617919-15814-0/minicluster-data/master-0/data/instance:
uuid: "28c834d3bd314a44b6f7228db138cd58"
format_stamp: "Formatted at 2026-08-09 22:58:01 on dist-test-slave-ncp9"
server_key: "6763f4d312a7534bc357c5ae5c1c9345"
server_key_iv: "e985ceab938f2b18046f01a2727fd9cf"
server_key_version: "encryptionkey@0"
I20260809 22:58:01.903497 16632 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786315935617919-15814-0/minicluster-data/master-0/wal/instance:
uuid: "28c834d3bd314a44b6f7228db138cd58"
format_stamp: "Formatted at 2026-08-09 22:58:01 on dist-test-slave-ncp9"
server_key: "6763f4d312a7534bc357c5ae5c1c9345"
server_key_iv: "e985ceab938f2b18046f01a2727fd9cf"
server_key_version: "encryptionkey@0"
I20260809 22:58:01.908330 16632 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260809 22:58:01.911566 16659 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:58:01.912799 16632 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 22:58:01.912990 16632 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786315935617919-15814-0/minicluster-data/master-0/data,/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786315935617919-15814-0/minicluster-data/master-0/wal
uuid: "28c834d3bd314a44b6f7228db138cd58"
format_stamp: "Formatted at 2026-08-09 22:58:01 on dist-test-slave-ncp9"
server_key: "6763f4d312a7534bc357c5ae5c1c9345"
server_key_iv: "e985ceab938f2b18046f01a2727fd9cf"
server_key_version: "encryptionkey@0"
I20260809 22:58:01.913166 16632 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786315935617919-15814-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786315935617919-15814-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786315935617919-15814-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:58:01.936944 16632 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:58:01.940914 16632 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:58:01.941171 16632 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:58:01.957983 16632 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42383
I20260809 22:58:01.958017 16739 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42383 every 8 connection(s)
I20260809 22:58:01.959223 16632 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786315935617919-15814-0/minicluster-data/master-0/data/info.pb
I20260809 22:58:01.960330 15814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskItYPJR/build/asan/bin/kudu as pid 16632
I20260809 22:58:01.960522 15814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786315935617919-15814-0/minicluster-data/master-0/wal/instance
I20260809 22:58:01.961134 15814 external_mini_cluster.cc:1428] Setting key 4d49def9388d7961e97def847636b96f
I20260809 22:58:01.963564 16741 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:58:01.973930 16744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:58:01.962851 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.213:46525 (local address 127.0.0.1:42383)
0809 22:58:01.963574 (+ 723us) server_negotiation.cc:206] Beginning negotiation
0809 22:58:01.963585 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 22:58:01.963624 (+ 39us) server_negotiation.cc:407] Connection header received
0809 22:58:01.965338 (+ 1714us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:58:01.965360 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:58:01.965772 (+ 412us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:58:01.966088 (+ 316us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:58:01.966696 (+ 608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:58:01.967964 (+ 1268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:58:01.969792 (+ 1828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:58:01.970412 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:58:01.970540 (+ 128us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:58:01.971759 (+ 1219us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:58:01.971766 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:58:01.971778 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 22:58:01.972298 (+ 520us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:58:01.972372 (+ 74us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 22:58:01.972395 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:58:01.972513 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0809 22:58:01.972817 (+ 304us) server_negotiation.cc:299] Negotiation successful
0809 22:58:01.973024 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":526,"thread_start_us":319,"threads_started":1}
I20260809 22:58:01.975566 16741 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 28c834d3bd314a44b6f7228db138cd58: Bootstrap starting.
I20260809 22:58:01.977622 15814 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 22:58:01.978757 16741 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 28c834d3bd314a44b6f7228db138cd58: Neither blocks nor log segments found. Creating new log.
I20260809 22:58:01.979701 16741 log.cc:824] T 00000000000000000000000000000000 P 28c834d3bd314a44b6f7228db138cd58: Log is configured to *not* fsync() on all Append() calls
I20260809 22:58:01.982823 16741 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 28c834d3bd314a44b6f7228db138cd58: No bootstrap required, opened a new log
I20260809 22:58:01.987749 16744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:58:01.981858 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.213:49575 (local address 127.0.0.1:42383)
0809 22:58:01.981992 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0809 22:58:01.981998 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:58:01.982450 (+ 452us) server_negotiation.cc:407] Connection header received
0809 22:58:01.982610 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:58:01.982614 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:58:01.982672 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:58:01.982749 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:58:01.983298 (+ 549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:58:01.984043 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:58:01.985371 (+ 1328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:58:01.985838 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:58:01.985951 (+ 113us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:58:01.987053 (+ 1102us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:58:01.987058 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:58:01.987061 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 22:58:01.987077 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:58:01.987112 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 22:58:01.987126 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:58:01.987230 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0809 22:58:01.987430 (+ 200us) server_negotiation.cc:299] Negotiation successful
0809 22:58:01.987488 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260809 22:58:01.990880 16741 raft_consensus.cc:348] T 00000000000000000000000000000000 P 28c834d3bd314a44b6f7228db138cd58 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28c834d3bd314a44b6f7228db138cd58" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42383 } }
I20260809 22:58:01.991243 16741 raft_consensus.cc:374] T 00000000000000000000000000000000 P 28c834d3bd314a44b6f7228db138cd58 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:58:01.991335 16741 raft_consensus.cc:729] T 00000000000000000000000000000000 P 28c834d3bd314a44b6f7228db138cd58 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 28c834d3bd314a44b6f7228db138cd58, State: Initialized, Role: FOLLOWER
I20260809 22:58:01.991993 16741 consensus_queue.cc:260] T 00000000000000000000000000000000 P 28c834d3bd314a44b6f7228db138cd58 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28c834d3bd314a44b6f7228db138cd58" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42383 } }
I20260809 22:58:01.992280 16741 raft_consensus.cc:388] T 00000000000000000000000000000000 P 28c834d3bd314a44b6f7228db138cd58 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 22:58:01.992431 16741 raft_consensus.cc:482] T 00000000000000000000000000000000 P 28c834d3bd314a44b6f7228db138cd58 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 22:58:01.992617 16741 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 28c834d3bd314a44b6f7228db138cd58 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:58:01.994763 16741 raft_consensus.cc:504] T 00000000000000000000000000000000 P 28c834d3bd314a44b6f7228db138cd58 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28c834d3bd314a44b6f7228db138cd58" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42383 } }
I20260809 22:58:01.995208 16741 leader_election.cc:302] T 00000000000000000000000000000000 P 28c834d3bd314a44b6f7228db138cd58 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 28c834d3bd314a44b6f7228db138cd58; no voters:
I20260809 22:58:01.995740 16741 leader_election.cc:288] T 00000000000000000000000000000000 P 28c834d3bd314a44b6f7228db138cd58 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 22:58:01.995817 16754 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 28c834d3bd314a44b6f7228db138cd58 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 22:58:01.996500 16754 raft_consensus.cc:686] T 00000000000000000000000000000000 P 28c834d3bd314a44b6f7228db138cd58 [term 1 LEADER]: Becoming Leader. State: Replica: 28c834d3bd314a44b6f7228db138cd58, State: Running, Role: LEADER
I20260809 22:58:01.997095 16754 consensus_queue.cc:237] T 00000000000000000000000000000000 P 28c834d3bd314a44b6f7228db138cd58 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28c834d3bd314a44b6f7228db138cd58" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42383 } }
I20260809 22:58:01.997925 16741 sys_catalog.cc:564] T 00000000000000000000000000000000 P 28c834d3bd314a44b6f7228db138cd58 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 22:58:02.000053 16755 sys_catalog.cc:455] T 00000000000000000000000000000000 P 28c834d3bd314a44b6f7228db138cd58 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "28c834d3bd314a44b6f7228db138cd58" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28c834d3bd314a44b6f7228db138cd58" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42383 } } }
I20260809 22:58:02.000520 16755 sys_catalog.cc:458] T 00000000000000000000000000000000 P 28c834d3bd314a44b6f7228db138cd58 [sys.catalog]: This master's current role is: LEADER
I20260809 22:58:02.000027 16758 sys_catalog.cc:455] T 00000000000000000000000000000000 P 28c834d3bd314a44b6f7228db138cd58 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 28c834d3bd314a44b6f7228db138cd58. Latest consensus state: current_term: 1 leader_uuid: "28c834d3bd314a44b6f7228db138cd58" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28c834d3bd314a44b6f7228db138cd58" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42383 } } }
I20260809 22:58:02.004933 16758 sys_catalog.cc:458] T 00000000000000000000000000000000 P 28c834d3bd314a44b6f7228db138cd58 [sys.catalog]: This master's current role is: LEADER
I20260809 22:58:02.005839 16765 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 22:58:02.010097 16765 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 22:58:02.018463 16765 catalog_manager.cc:1347] Generated new cluster ID: 9399217ab7994ec3bdd4b1c897f27ffa
I20260809 22:58:02.018566 16765 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 22:58:02.026258 16765 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 22:58:02.027405 16765 catalog_manager.cc:1504] Loading token signing keys...
I20260809 22:58:02.034446 16765 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 28c834d3bd314a44b6f7228db138cd58: Generated new TSK 0
I20260809 22:58:02.035164 16765 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 22:58:02.041203 15814 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskItYPJR/build/asan/bin/kudu pid 16632
W20260809 22:58:02.042349 16734 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.213:46525
W20260809 22:58:02.043097 15814 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskItYPJR/build/asan/bin/kudu pid 16632: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 22:58:02.043246 15814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskItYPJR/build/asan/bin/kudu with pid 16632
W20260809 22:58:02.064560 16609 connection.cc:535] client connection to 127.0.0.1:42383 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:42383): Connection reset by peer (error 104)
2026-08-09T22:58:02Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (2050 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786315935617919-15814-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 22:58:02 dist-test-slave-ncp9 krb5kdc[16786](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 22:58:02 dist-test-slave-ncp9 krb5kdc[16786](info): set up 2 sockets
Aug 09 22:58:02 dist-test-slave-ncp9 krb5kdc[16786](info): commencing operation
krb5kdc: starting...
W20260809 22:58:03.251010 15814 mini_kdc.cc:121] Time spent starting KDC: real 1.168s user 0.006s sys 0.040s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786315935617919-15814-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786315935617919-15814-0/krb5kdc/test-user.keytab.
Aug 09 22:58:03 dist-test-slave-ncp9 krb5kdc[16786](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316283, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T22:58:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T22:58:03Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786315935617919-15814-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786315935617919-15814-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 22:58:03.539999 15814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskItYPJR/build/asan/bin/kudu
/tmp/dist-test-taskItYPJR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786315935617919-15814-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786315935617919-15814-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786315935617919-15814-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786315935617919-15814-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41369
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786315935617919-15814-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36503
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41369
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786315935617919-15814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786315935617919-15814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786315935617919-15814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:58:03.870702 16803 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:58:03.870965 16803 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:58:03.871095 16803 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:58:03.878003 16803 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 22:58:03.878103 16803 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:58:03.878177 16803 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:58:03.878253 16803 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 22:58:03.878312 16803 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 22:58:03.887621 16803 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36503
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786315935617919-15814-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786315935617919-15814-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41369
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786315935617919-15814-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41369
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786315935617919-15814-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786315935617919-15814-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:58:03.888837 16803 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:58:03.890128 16803 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:58:03.901759 16813 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:58:03.901788 16817 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:58:03.901714 16814 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:58:03.902503 16803 server_base.cc:1034] running on GCE node
I20260809 22:58:03.903198 16803 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:58:03.904476 16803 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:58:03.905678 16803 hybrid_clock.cc:648] HybridClock initialized: now 1786316283905630 us; error 35 us; skew 500 ppm
I20260809 22:58:03.905997 16803 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:58:03 dist-test-slave-ncp9 krb5kdc[16786](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316283, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:58:03.908864 16803 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 22:58:03.910192 16803 webserver.cc:460] Webserver started at http://127.0.0.1:34509/ using document root <none> and password file <none>
I20260809 22:58:03.910831 16803 fs_manager.cc:359] Metadata directory not provided
I20260809 22:58:03.910948 16803 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:58:03.911230 16803 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:58:03.914151 16803 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786315935617919-15814-0/minicluster-data/master-0/data/instance:
uuid: "e5479f3076564cf29671c939492f276c"
format_stamp: "Formatted at 2026-08-09 22:58:03 on dist-test-slave-ncp9"
server_key: "47c2062df4cb9abfe703122fde7e354e"
server_key_iv: "70277a680e9a7dadc6bff4151257217f"
server_key_version: "encryptionkey@0"
I20260809 22:58:03.914894 16803 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786315935617919-15814-0/minicluster-data/master-0/wal/instance:
uuid: "e5479f3076564cf29671c939492f276c"
format_stamp: "Formatted at 2026-08-09 22:58:03 on dist-test-slave-ncp9"
server_key: "47c2062df4cb9abfe703122fde7e354e"
server_key_iv: "70277a680e9a7dadc6bff4151257217f"
server_key_version: "encryptionkey@0"
I20260809 22:58:03.919759 16803 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260809 22:58:03.923069 16830 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:58:03.924259 16803 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260809 22:58:03.924418 16803 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786315935617919-15814-0/minicluster-data/master-0/data,/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786315935617919-15814-0/minicluster-data/master-0/wal
uuid: "e5479f3076564cf29671c939492f276c"
format_stamp: "Formatted at 2026-08-09 22:58:03 on dist-test-slave-ncp9"
server_key: "47c2062df4cb9abfe703122fde7e354e"
server_key_iv: "70277a680e9a7dadc6bff4151257217f"
server_key_version: "encryptionkey@0"
I20260809 22:58:03.924589 16803 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786315935617919-15814-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786315935617919-15814-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786315935617919-15814-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:58:03.930017 16803 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:58:03.933589 16803 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:58:03.933856 16803 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:58:03.950281 16803 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41369
I20260809 22:58:03.950325 16908 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41369 every 8 connection(s)
I20260809 22:58:03.951530 16803 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786315935617919-15814-0/minicluster-data/master-0/data/info.pb
I20260809 22:58:03.955015 16910 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:58:03.958024 15814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskItYPJR/build/asan/bin/kudu as pid 16803
I20260809 22:58:03.958190 15814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786315935617919-15814-0/minicluster-data/master-0/wal/instance
I20260809 22:58:03.958604 15814 external_mini_cluster.cc:1428] Setting key 6de82c07dee1b095cd293805f4541f64
I20260809 22:58:03.965801 16916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:58:03.960210 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.213:33655 (local address 127.0.0.1:41369)
0809 22:58:03.960974 (+ 764us) server_negotiation.cc:206] Beginning negotiation
0809 22:58:03.960985 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 22:58:03.961027 (+ 42us) server_negotiation.cc:407] Connection header received
0809 22:58:03.962634 (+ 1607us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:58:03.962656 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:58:03.962987 (+ 331us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:58:03.963294 (+ 307us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:58:03.964304 (+ 1010us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:58:03.964310 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:58:03.964326 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 22:58:03.964339 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:58:03.964397 (+ 58us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 22:58:03.964416 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:58:03.964578 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0809 22:58:03.964707 (+ 129us) server_negotiation.cc:299] Negotiation successful
0809 22:58:03.964881 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":524,"thread_start_us":303,"threads_started":1}
I20260809 22:58:03.966464 16910 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e5479f3076564cf29671c939492f276c: Bootstrap starting.
I20260809 22:58:03.969375 15814 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 22:58:03.969753 16910 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e5479f3076564cf29671c939492f276c: Neither blocks nor log segments found. Creating new log.
I20260809 22:58:03.970918 16910 log.cc:824] T 00000000000000000000000000000000 P e5479f3076564cf29671c939492f276c: Log is configured to *not* fsync() on all Append() calls
I20260809 22:58:03.973964 16910 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e5479f3076564cf29671c939492f276c: No bootstrap required, opened a new log
I20260809 22:58:03.976568 16916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:58:03.974270 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.213:36587 (local address 127.0.0.1:41369)
0809 22:58:03.974468 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0809 22:58:03.974474 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:58:03.974829 (+ 355us) server_negotiation.cc:407] Connection header received
0809 22:58:03.974977 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:58:03.974980 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:58:03.975020 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:58:03.975091 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:58:03.975977 (+ 886us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:58:03.975981 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:58:03.975984 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 22:58:03.975997 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:58:03.976035 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 22:58:03.976051 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:58:03.976143 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0809 22:58:03.976267 (+ 124us) server_negotiation.cc:299] Negotiation successful
0809 22:58:03.976314 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260809 22:58:03.981995 16910 raft_consensus.cc:348] T 00000000000000000000000000000000 P e5479f3076564cf29671c939492f276c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5479f3076564cf29671c939492f276c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41369 } }
I20260809 22:58:03.982335 16910 raft_consensus.cc:374] T 00000000000000000000000000000000 P e5479f3076564cf29671c939492f276c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:58:03.982429 16910 raft_consensus.cc:729] T 00000000000000000000000000000000 P e5479f3076564cf29671c939492f276c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5479f3076564cf29671c939492f276c, State: Initialized, Role: FOLLOWER
I20260809 22:58:03.982935 16910 consensus_queue.cc:260] T 00000000000000000000000000000000 P e5479f3076564cf29671c939492f276c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5479f3076564cf29671c939492f276c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41369 } }
I20260809 22:58:03.983142 16910 raft_consensus.cc:388] T 00000000000000000000000000000000 P e5479f3076564cf29671c939492f276c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 22:58:03.983273 16910 raft_consensus.cc:482] T 00000000000000000000000000000000 P e5479f3076564cf29671c939492f276c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 22:58:03.983423 16910 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e5479f3076564cf29671c939492f276c [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:58:03.985745 16910 raft_consensus.cc:504] T 00000000000000000000000000000000 P e5479f3076564cf29671c939492f276c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5479f3076564cf29671c939492f276c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41369 } }
I20260809 22:58:03.986203 16910 leader_election.cc:302] T 00000000000000000000000000000000 P e5479f3076564cf29671c939492f276c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e5479f3076564cf29671c939492f276c; no voters:
I20260809 22:58:03.986729 16910 leader_election.cc:288] T 00000000000000000000000000000000 P e5479f3076564cf29671c939492f276c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 22:58:03.986832 16926 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e5479f3076564cf29671c939492f276c [term 1 FOLLOWER]: Leader election won for term 1
I20260809 22:58:03.987426 16926 raft_consensus.cc:686] T 00000000000000000000000000000000 P e5479f3076564cf29671c939492f276c [term 1 LEADER]: Becoming Leader. State: Replica: e5479f3076564cf29671c939492f276c, State: Running, Role: LEADER
I20260809 22:58:03.987974 16926 consensus_queue.cc:237] T 00000000000000000000000000000000 P e5479f3076564cf29671c939492f276c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5479f3076564cf29671c939492f276c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41369 } }
I20260809 22:58:03.988587 16910 sys_catalog.cc:564] T 00000000000000000000000000000000 P e5479f3076564cf29671c939492f276c [sys.catalog]: configured and running, proceeding with master startup.
I20260809 22:58:03.990773 16929 sys_catalog.cc:455] T 00000000000000000000000000000000 P e5479f3076564cf29671c939492f276c [sys.catalog]: SysCatalogTable state changed. Reason: New leader e5479f3076564cf29671c939492f276c. Latest consensus state: current_term: 1 leader_uuid: "e5479f3076564cf29671c939492f276c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5479f3076564cf29671c939492f276c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41369 } } }
I20260809 22:58:03.991194 16929 sys_catalog.cc:458] T 00000000000000000000000000000000 P e5479f3076564cf29671c939492f276c [sys.catalog]: This master's current role is: LEADER
I20260809 22:58:03.990723 16928 sys_catalog.cc:455] T 00000000000000000000000000000000 P e5479f3076564cf29671c939492f276c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e5479f3076564cf29671c939492f276c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5479f3076564cf29671c939492f276c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41369 } } }
I20260809 22:58:03.992166 16928 sys_catalog.cc:458] T 00000000000000000000000000000000 P e5479f3076564cf29671c939492f276c [sys.catalog]: This master's current role is: LEADER
I20260809 22:58:03.995242 16935 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 22:58:04.000218 16935 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 22:58:04.008445 16935 catalog_manager.cc:1347] Generated new cluster ID: 7851fe2d700e4c7f8e6f76be6be0124d
I20260809 22:58:04.008554 16935 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 22:58:04.032387 16935 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 22:58:04.032919 16935 catalog_manager.cc:1504] Loading token signing keys...
I20260809 22:58:04.042449 16935 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e5479f3076564cf29671c939492f276c: Generated new TSK 0
I20260809 22:58:04.043184 16935 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 22:58:04.049132 15814 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskItYPJR/build/asan/bin/kudu pid 16803
W20260809 22:58:04.050232 16903 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.213:33655
W20260809 22:58:04.050902 15814 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskItYPJR/build/asan/bin/kudu pid 16803: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 22:58:04.051036 15814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskItYPJR/build/asan/bin/kudu with pid 16803
2026-08-09T22:58:04Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (2012 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786315935617919-15814-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 22:58:04 dist-test-slave-ncp9 krb5kdc[16962](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 22:58:04 dist-test-slave-ncp9 krb5kdc[16962](info): set up 2 sockets
Aug 09 22:58:04 dist-test-slave-ncp9 krb5kdc[16962](info): commencing operation
krb5kdc: starting...
W20260809 22:58:05.282541 15814 mini_kdc.cc:121] Time spent starting KDC: real 1.187s user 0.001s sys 0.042s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786315935617919-15814-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786315935617919-15814-0/krb5kdc/test-user.keytab.
Aug 09 22:58:05 dist-test-slave-ncp9 krb5kdc[16962](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316285, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T22:58:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T22:58:05Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.113.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.113.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.113.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.190.keytab.
Entry for principal kudu/127.15.113.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.190.keytab.
I20260809 22:58:05.584188 15814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskItYPJR/build/asan/bin/kudu
/tmp/dist-test-taskItYPJR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786315935617919-15814-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786315935617919-15814-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786315935617919-15814-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786315935617919-15814-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.113.190:45495
--webserver_interface=127.15.113.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.190.keytab
--principal=kudu/127.15.113.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.113.148:42667
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.113.190:45495
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786315935617919-15814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786315935617919-15814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786315935617919-15814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:58:05.920941 16982 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:58:05.921204 16982 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:58:05.921339 16982 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:58:05.928535 16982 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 22:58:05.928656 16982 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:58:05.928735 16982 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:58:05.928812 16982 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 22:58:05.928879 16982 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 22:58:05.938582 16982 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.113.148:42667
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786315935617919-15814-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786315935617919-15814-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.113.190:45495
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.190.keytab
--principal=kudu/127.15.113.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.113.190:45495
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786315935617919-15814-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.113.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786315935617919-15814-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:58:05.939808 16982 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:58:05.941102 16982 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:58:05.952251 16991 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:58:05.952275 16993 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:58:05.952329 16989 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:58:05.952961 16982 server_base.cc:1034] running on GCE node
I20260809 22:58:05.953727 16982 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:58:05.955016 16982 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:58:05.956233 16982 hybrid_clock.cc:648] HybridClock initialized: now 1786316285956181 us; error 42 us; skew 500 ppm
I20260809 22:58:05.956569 16982 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:58:05 dist-test-slave-ncp9 krb5kdc[16962](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316285, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.113.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:58:05.963258 16982 init.cc:377] Logged in from keytab as kudu/127.15.113.190@KRBTEST.COM (short username kudu)
I20260809 22:58:05.964660 16982 webserver.cc:460] Webserver started at http://127.15.113.190:34677/ using document root <none> and password file <none>
I20260809 22:58:05.965354 16982 fs_manager.cc:359] Metadata directory not provided
I20260809 22:58:05.965471 16982 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:58:05.965775 16982 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:58:05.968669 16982 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786315935617919-15814-0/minicluster-data/master-0/data/instance:
uuid: "d9e4807effef426782453fc3937d005a"
format_stamp: "Formatted at 2026-08-09 22:58:05 on dist-test-slave-ncp9"
server_key: "c515f158b7079c36b072afcdd60c16a6"
server_key_iv: "64bea991d25e3da7870b97a99da4ca18"
server_key_version: "encryptionkey@0"
I20260809 22:58:05.969470 16982 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786315935617919-15814-0/minicluster-data/master-0/wal/instance:
uuid: "d9e4807effef426782453fc3937d005a"
format_stamp: "Formatted at 2026-08-09 22:58:05 on dist-test-slave-ncp9"
server_key: "c515f158b7079c36b072afcdd60c16a6"
server_key_iv: "64bea991d25e3da7870b97a99da4ca18"
server_key_version: "encryptionkey@0"
I20260809 22:58:05.974474 16982 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260809 22:58:05.977787 17002 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:58:05.979005 16982 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 22:58:05.979161 16982 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786315935617919-15814-0/minicluster-data/master-0/data,/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786315935617919-15814-0/minicluster-data/master-0/wal
uuid: "d9e4807effef426782453fc3937d005a"
format_stamp: "Formatted at 2026-08-09 22:58:05 on dist-test-slave-ncp9"
server_key: "c515f158b7079c36b072afcdd60c16a6"
server_key_iv: "64bea991d25e3da7870b97a99da4ca18"
server_key_version: "encryptionkey@0"
I20260809 22:58:05.979341 16982 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786315935617919-15814-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786315935617919-15814-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786315935617919-15814-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:58:05.994910 16982 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:58:05.998914 16982 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:58:05.999164 16982 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:58:06.016350 16982 rpc_server.cc:307] RPC server started. Bound to: 127.15.113.190:45495
I20260809 22:58:06.016376 17075 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.113.190:45495 every 8 connection(s)
I20260809 22:58:06.017697 16982 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786315935617919-15814-0/minicluster-data/master-0/data/info.pb
I20260809 22:58:06.020869 15814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskItYPJR/build/asan/bin/kudu as pid 16982
I20260809 22:58:06.021062 15814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786315935617919-15814-0/minicluster-data/master-0/wal/instance
I20260809 22:58:06.021485 15814 external_mini_cluster.cc:1428] Setting key ef3fdb729d2db61c9a5885e7fc263c8c
I20260809 22:58:06.022120 17076 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:58:06.033906 17076 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d9e4807effef426782453fc3937d005a: Bootstrap starting.
Aug 09 22:58:06 dist-test-slave-ncp9 krb5kdc[16962](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316285, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.113.190@KRBTEST.COM
I20260809 22:58:06.037235 17076 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d9e4807effef426782453fc3937d005a: Neither blocks nor log segments found. Creating new log.
I20260809 22:58:06.038228 17076 log.cc:824] T 00000000000000000000000000000000 P d9e4807effef426782453fc3937d005a: Log is configured to *not* fsync() on all Append() calls
I20260809 22:58:06.040745 17076 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d9e4807effef426782453fc3937d005a: No bootstrap required, opened a new log
I20260809 22:58:06.042282 17080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:58:06.023178 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48239 (local address 127.15.113.190:45495)
0809 22:58:06.023923 (+ 745us) server_negotiation.cc:206] Beginning negotiation
0809 22:58:06.023935 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 22:58:06.023973 (+ 38us) server_negotiation.cc:407] Connection header received
0809 22:58:06.025541 (+ 1568us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:58:06.025562 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:58:06.025947 (+ 385us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:58:06.026295 (+ 348us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:58:06.027731 (+ 1436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:58:06.029074 (+ 1343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:58:06.030293 (+ 1219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:58:06.030957 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:58:06.031133 (+ 176us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:58:06.035486 (+ 4353us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:58:06.035494 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:58:06.035507 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:58:06.035544 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:58:06.038475 (+ 2931us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:58:06.039185 (+ 710us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:58:06.039191 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:58:06.039197 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:58:06.039347 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:58:06.039786 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:58:06.039790 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:58:06.039792 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:58:06.040278 (+ 486us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:58:06.040559 (+ 281us) server_negotiation.cc:1035] Waiting for connection context
0809 22:58:06.041073 (+ 514us) server_negotiation.cc:299] Negotiation successful
0809 22:58:06.041364 (+ 291us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":551,"thread_start_us":328,"threads_started":1}
I20260809 22:58:06.046114 15814 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 22:58:06.048978 17076 raft_consensus.cc:348] T 00000000000000000000000000000000 P d9e4807effef426782453fc3937d005a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d9e4807effef426782453fc3937d005a" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 45495 } }
I20260809 22:58:06.049296 17076 raft_consensus.cc:374] T 00000000000000000000000000000000 P d9e4807effef426782453fc3937d005a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:58:06.049386 17076 raft_consensus.cc:729] T 00000000000000000000000000000000 P d9e4807effef426782453fc3937d005a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d9e4807effef426782453fc3937d005a, State: Initialized, Role: FOLLOWER
I20260809 22:58:06.049948 17076 consensus_queue.cc:260] T 00000000000000000000000000000000 P d9e4807effef426782453fc3937d005a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d9e4807effef426782453fc3937d005a" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 45495 } }
I20260809 22:58:06.050277 17076 raft_consensus.cc:388] T 00000000000000000000000000000000 P d9e4807effef426782453fc3937d005a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 22:58:06.050446 17076 raft_consensus.cc:482] T 00000000000000000000000000000000 P d9e4807effef426782453fc3937d005a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 22:58:06.050635 17076 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d9e4807effef426782453fc3937d005a [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:58:06.052700 17076 raft_consensus.cc:504] T 00000000000000000000000000000000 P d9e4807effef426782453fc3937d005a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d9e4807effef426782453fc3937d005a" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 45495 } }
I20260809 22:58:06.053220 17076 leader_election.cc:302] T 00000000000000000000000000000000 P d9e4807effef426782453fc3937d005a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d9e4807effef426782453fc3937d005a; no voters:
I20260809 22:58:06.053789 17076 leader_election.cc:288] T 00000000000000000000000000000000 P d9e4807effef426782453fc3937d005a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 22:58:06.053900 17092 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d9e4807effef426782453fc3937d005a [term 1 FOLLOWER]: Leader election won for term 1
I20260809 22:58:06.054953 17092 raft_consensus.cc:686] T 00000000000000000000000000000000 P d9e4807effef426782453fc3937d005a [term 1 LEADER]: Becoming Leader. State: Replica: d9e4807effef426782453fc3937d005a, State: Running, Role: LEADER
I20260809 22:58:06.055550 17092 consensus_queue.cc:237] T 00000000000000000000000000000000 P d9e4807effef426782453fc3937d005a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d9e4807effef426782453fc3937d005a" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 45495 } }
I20260809 22:58:06.055959 17076 sys_catalog.cc:564] T 00000000000000000000000000000000 P d9e4807effef426782453fc3937d005a [sys.catalog]: configured and running, proceeding with master startup.
I20260809 22:58:06.058215 17096 sys_catalog.cc:455] T 00000000000000000000000000000000 P d9e4807effef426782453fc3937d005a [sys.catalog]: SysCatalogTable state changed. Reason: New leader d9e4807effef426782453fc3937d005a. Latest consensus state: current_term: 1 leader_uuid: "d9e4807effef426782453fc3937d005a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d9e4807effef426782453fc3937d005a" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 45495 } } }
I20260809 22:58:06.059227 17096 sys_catalog.cc:458] T 00000000000000000000000000000000 P d9e4807effef426782453fc3937d005a [sys.catalog]: This master's current role is: LEADER
I20260809 22:58:06.059641 17095 sys_catalog.cc:455] T 00000000000000000000000000000000 P d9e4807effef426782453fc3937d005a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d9e4807effef426782453fc3937d005a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d9e4807effef426782453fc3937d005a" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 45495 } } }
I20260809 22:58:06.060214 17095 sys_catalog.cc:458] T 00000000000000000000000000000000 P d9e4807effef426782453fc3937d005a [sys.catalog]: This master's current role is: LEADER
I20260809 22:58:06.064213 17101 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 22:58:06.067432 17080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:58:06.050110 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57003 (local address 127.15.113.190:45495)
0809 22:58:06.050257 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0809 22:58:06.050266 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 22:58:06.050858 (+ 592us) server_negotiation.cc:407] Connection header received
0809 22:58:06.051012 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:58:06.051017 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:58:06.051077 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:58:06.051176 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:58:06.052496 (+ 1320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:58:06.053353 (+ 857us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:58:06.054854 (+ 1501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:58:06.055449 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:58:06.055520 (+ 71us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:58:06.058388 (+ 2868us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:58:06.058395 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:58:06.058400 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:58:06.058439 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:58:06.063624 (+ 5185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:58:06.064327 (+ 703us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:58:06.064332 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:58:06.064335 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:58:06.064539 (+ 204us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:58:06.065097 (+ 558us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:58:06.065104 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:58:06.065106 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:58:06.065375 (+ 269us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:58:06.065666 (+ 291us) server_negotiation.cc:1035] Waiting for connection context
0809 22:58:06.066900 (+ 1234us) server_negotiation.cc:299] Negotiation successful
0809 22:58:06.067071 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260809 22:58:06.070569 17101 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 22:58:06.079129 17101 catalog_manager.cc:1347] Generated new cluster ID: 28f8a2ee8cf643b39e43e4348ed4d72e
I20260809 22:58:06.079236 17101 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 22:58:06.097995 17101 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 22:58:06.099151 17101 catalog_manager.cc:1504] Loading token signing keys...
I20260809 22:58:06.105906 17101 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d9e4807effef426782453fc3937d005a: Generated new TSK 0
I20260809 22:58:06.106613 17101 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 22:58:06.115069 15814 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskItYPJR/build/asan/bin/kudu pid 16982
I20260809 22:58:06.116312 17068 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:58:06.205439 15814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskItYPJR/build/asan/bin/kudu with pid 16982
W20260809 22:58:06.228405 16956 connection.cc:535] client connection to 127.15.113.190:45495 recv error: Network error: failed to read from TLS socket (remote: 127.15.113.190:45495): Connection reset by peer (error 104)
2026-08-09T22:58:06Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (2151 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786315935617919-15814-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 22:58:06 dist-test-slave-ncp9 krb5kdc[17129](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 22:58:06 dist-test-slave-ncp9 krb5kdc[17129](info): set up 2 sockets
Aug 09 22:58:06 dist-test-slave-ncp9 krb5kdc[17129](info): commencing operation
krb5kdc: starting...
W20260809 22:58:07.408325 15814 mini_kdc.cc:121] Time spent starting KDC: real 1.162s user 0.004s sys 0.037s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786315935617919-15814-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786315935617919-15814-0/krb5kdc/test-user.keytab.
Aug 09 22:58:07 dist-test-slave-ncp9 krb5kdc[17129](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316287, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T22:58:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T22:58:07Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.113.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.113.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.113.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.190.keytab.
Entry for principal kudu/127.15.113.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.190.keytab.
I20260809 22:58:07.751879 15814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskItYPJR/build/asan/bin/kudu
/tmp/dist-test-taskItYPJR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786315935617919-15814-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786315935617919-15814-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786315935617919-15814-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786315935617919-15814-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.113.190:35511
--webserver_interface=127.15.113.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.190.keytab
--principal=kudu/127.15.113.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.113.148:39305
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.113.190:35511
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786315935617919-15814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786315935617919-15814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786315935617919-15814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:58:08.082489 17155 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:58:08.082775 17155 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:58:08.082875 17155 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:58:08.089975 17155 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 22:58:08.090082 17155 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:58:08.090135 17155 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:58:08.090195 17155 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 22:58:08.090238 17155 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 22:58:08.099819 17155 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.113.148:39305
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786315935617919-15814-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786315935617919-15814-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.113.190:35511
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.190.keytab
--principal=kudu/127.15.113.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.113.190:35511
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786315935617919-15814-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.113.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786315935617919-15814-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:58:08.101037 17155 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:58:08.102350 17155 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:58:08.113503 17160 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:58:08.113420 17163 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:58:08.113466 17161 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:58:08.114374 17155 server_base.cc:1034] running on GCE node
I20260809 22:58:08.115067 17155 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:58:08.116341 17155 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:58:08.117538 17155 hybrid_clock.cc:648] HybridClock initialized: now 1786316288117492 us; error 37 us; skew 500 ppm
I20260809 22:58:08.117867 17155 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:58:08 dist-test-slave-ncp9 krb5kdc[17129](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316288, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.113.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:58:08.120795 17155 init.cc:377] Logged in from keytab as kudu/127.15.113.190@KRBTEST.COM (short username kudu)
I20260809 22:58:08.122205 17155 webserver.cc:460] Webserver started at http://127.15.113.190:34685/ using document root <none> and password file <none>
I20260809 22:58:08.122850 17155 fs_manager.cc:359] Metadata directory not provided
I20260809 22:58:08.122969 17155 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:58:08.123267 17155 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:58:08.126297 17155 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786315935617919-15814-0/minicluster-data/master-0/data/instance:
uuid: "05649e26791540adb83bba7836ebfd26"
format_stamp: "Formatted at 2026-08-09 22:58:08 on dist-test-slave-ncp9"
server_key: "fcf4a51b52ef6b342d41758b07bb7cd7"
server_key_iv: "33eb2cf3b1cbced15e2d0a4677a92745"
server_key_version: "encryptionkey@0"
I20260809 22:58:08.127036 17155 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786315935617919-15814-0/minicluster-data/master-0/wal/instance:
uuid: "05649e26791540adb83bba7836ebfd26"
format_stamp: "Formatted at 2026-08-09 22:58:08 on dist-test-slave-ncp9"
server_key: "fcf4a51b52ef6b342d41758b07bb7cd7"
server_key_iv: "33eb2cf3b1cbced15e2d0a4677a92745"
server_key_version: "encryptionkey@0"
I20260809 22:58:08.131992 17155 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.004s
I20260809 22:58:08.135358 17173 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:58:08.136592 17155 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 22:58:08.136744 17155 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786315935617919-15814-0/minicluster-data/master-0/data,/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786315935617919-15814-0/minicluster-data/master-0/wal
uuid: "05649e26791540adb83bba7836ebfd26"
format_stamp: "Formatted at 2026-08-09 22:58:08 on dist-test-slave-ncp9"
server_key: "fcf4a51b52ef6b342d41758b07bb7cd7"
server_key_iv: "33eb2cf3b1cbced15e2d0a4677a92745"
server_key_version: "encryptionkey@0"
I20260809 22:58:08.136937 17155 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786315935617919-15814-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786315935617919-15814-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786315935617919-15814-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:58:08.151983 17155 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:58:08.155741 17155 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:58:08.156002 17155 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:58:08.172483 17155 rpc_server.cc:307] RPC server started. Bound to: 127.15.113.190:35511
I20260809 22:58:08.172508 17247 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.113.190:35511 every 8 connection(s)
I20260809 22:58:08.173834 17155 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786315935617919-15814-0/minicluster-data/master-0/data/info.pb
I20260809 22:58:08.177176 15814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskItYPJR/build/asan/bin/kudu as pid 17155
I20260809 22:58:08.177366 15814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786315935617919-15814-0/minicluster-data/master-0/wal/instance
I20260809 22:58:08.177727 17248 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:58:08.177942 15814 external_mini_cluster.cc:1428] Setting key d6de8f3178c5411e076b5fa12d9156fd
I20260809 22:58:08.191063 17251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:58:08.179678 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53931 (local address 127.15.113.190:35511)
0809 22:58:08.180474 (+ 796us) server_negotiation.cc:206] Beginning negotiation
0809 22:58:08.180486 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 22:58:08.180529 (+ 43us) server_negotiation.cc:407] Connection header received
0809 22:58:08.182432 (+ 1903us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:58:08.182460 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:58:08.182902 (+ 442us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:58:08.183230 (+ 328us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:58:08.183781 (+ 551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:58:08.185180 (+ 1399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:58:08.186893 (+ 1713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:58:08.187535 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:58:08.187659 (+ 124us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:58:08.188944 (+ 1285us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:58:08.188955 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:58:08.188972 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 22:58:08.189506 (+ 534us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:58:08.189586 (+ 80us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 22:58:08.189615 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:58:08.189731 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0809 22:58:08.189975 (+ 244us) server_negotiation.cc:299] Negotiation successful
0809 22:58:08.190144 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":611,"thread_start_us":336,"threads_started":1}
I20260809 22:58:08.191090 17248 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 05649e26791540adb83bba7836ebfd26: Bootstrap starting.
I20260809 22:58:08.194463 17248 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 05649e26791540adb83bba7836ebfd26: Neither blocks nor log segments found. Creating new log.
I20260809 22:58:08.194922 15814 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 22:58:08.195645 17248 log.cc:824] T 00000000000000000000000000000000 P 05649e26791540adb83bba7836ebfd26: Log is configured to *not* fsync() on all Append() calls
I20260809 22:58:08.198783 17248 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 05649e26791540adb83bba7836ebfd26: No bootstrap required, opened a new log
I20260809 22:58:08.206171 17251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:58:08.199715 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50003 (local address 127.15.113.190:35511)
0809 22:58:08.199872 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0809 22:58:08.199877 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:58:08.200357 (+ 480us) server_negotiation.cc:407] Connection header received
0809 22:58:08.200479 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:58:08.200484 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:58:08.200561 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:58:08.200679 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:58:08.201142 (+ 463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:58:08.202305 (+ 1163us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:58:08.203561 (+ 1256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:58:08.204269 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:58:08.204399 (+ 130us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:58:08.205445 (+ 1046us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:58:08.205455 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:58:08.205459 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 22:58:08.205479 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:58:08.205538 (+ 59us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 22:58:08.205556 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:58:08.205669 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0809 22:58:08.205780 (+ 111us) server_negotiation.cc:299] Negotiation successful
0809 22:58:08.205833 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260809 22:58:08.208171 17248 raft_consensus.cc:348] T 00000000000000000000000000000000 P 05649e26791540adb83bba7836ebfd26 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "05649e26791540adb83bba7836ebfd26" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 35511 } }
I20260809 22:58:08.208551 17248 raft_consensus.cc:374] T 00000000000000000000000000000000 P 05649e26791540adb83bba7836ebfd26 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:58:08.208647 17248 raft_consensus.cc:729] T 00000000000000000000000000000000 P 05649e26791540adb83bba7836ebfd26 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 05649e26791540adb83bba7836ebfd26, State: Initialized, Role: FOLLOWER
I20260809 22:58:08.209276 17248 consensus_queue.cc:260] T 00000000000000000000000000000000 P 05649e26791540adb83bba7836ebfd26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "05649e26791540adb83bba7836ebfd26" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 35511 } }
I20260809 22:58:08.209570 17248 raft_consensus.cc:388] T 00000000000000000000000000000000 P 05649e26791540adb83bba7836ebfd26 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 22:58:08.209728 17248 raft_consensus.cc:482] T 00000000000000000000000000000000 P 05649e26791540adb83bba7836ebfd26 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 22:58:08.209892 17248 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 05649e26791540adb83bba7836ebfd26 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:58:08.211939 17248 raft_consensus.cc:504] T 00000000000000000000000000000000 P 05649e26791540adb83bba7836ebfd26 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "05649e26791540adb83bba7836ebfd26" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 35511 } }
I20260809 22:58:08.212384 17248 leader_election.cc:302] T 00000000000000000000000000000000 P 05649e26791540adb83bba7836ebfd26 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 05649e26791540adb83bba7836ebfd26; no voters:
I20260809 22:58:08.212904 17248 leader_election.cc:288] T 00000000000000000000000000000000 P 05649e26791540adb83bba7836ebfd26 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 22:58:08.213054 17263 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 05649e26791540adb83bba7836ebfd26 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 22:58:08.213730 17263 raft_consensus.cc:686] T 00000000000000000000000000000000 P 05649e26791540adb83bba7836ebfd26 [term 1 LEADER]: Becoming Leader. State: Replica: 05649e26791540adb83bba7836ebfd26, State: Running, Role: LEADER
I20260809 22:58:08.214304 17263 consensus_queue.cc:237] T 00000000000000000000000000000000 P 05649e26791540adb83bba7836ebfd26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "05649e26791540adb83bba7836ebfd26" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 35511 } }
I20260809 22:58:08.214859 17248 sys_catalog.cc:564] T 00000000000000000000000000000000 P 05649e26791540adb83bba7836ebfd26 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 22:58:08.217329 17265 sys_catalog.cc:455] T 00000000000000000000000000000000 P 05649e26791540adb83bba7836ebfd26 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 05649e26791540adb83bba7836ebfd26. Latest consensus state: current_term: 1 leader_uuid: "05649e26791540adb83bba7836ebfd26" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "05649e26791540adb83bba7836ebfd26" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 35511 } } }
I20260809 22:58:08.217304 17264 sys_catalog.cc:455] T 00000000000000000000000000000000 P 05649e26791540adb83bba7836ebfd26 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "05649e26791540adb83bba7836ebfd26" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "05649e26791540adb83bba7836ebfd26" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 35511 } } }
I20260809 22:58:08.217783 17265 sys_catalog.cc:458] T 00000000000000000000000000000000 P 05649e26791540adb83bba7836ebfd26 [sys.catalog]: This master's current role is: LEADER
I20260809 22:58:08.217785 17264 sys_catalog.cc:458] T 00000000000000000000000000000000 P 05649e26791540adb83bba7836ebfd26 [sys.catalog]: This master's current role is: LEADER
I20260809 22:58:08.219414 17271 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 22:58:08.224503 17271 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 22:58:08.233034 17271 catalog_manager.cc:1347] Generated new cluster ID: 7503f51af5f8419aaecb46ec1eb8d5c2
I20260809 22:58:08.233140 17271 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 22:58:08.242146 17271 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 22:58:08.243882 17271 catalog_manager.cc:1504] Loading token signing keys...
I20260809 22:58:08.254156 17271 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 05649e26791540adb83bba7836ebfd26: Generated new TSK 0
I20260809 22:58:08.255079 17271 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 22:58:08.268591 15814 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskItYPJR/build/asan/bin/kudu pid 17155
W20260809 22:58:08.270116 17242 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:53931
W20260809 22:58:08.270757 15814 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskItYPJR/build/asan/bin/kudu pid 17155: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 22:58:08.270917 15814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskItYPJR/build/asan/bin/kudu with pid 17155
W20260809 22:58:08.293308 17123 connection.cc:535] client connection to 127.15.113.190:35511 recv error: Network error: failed to read from TLS socket (remote: 127.15.113.190:35511): Connection reset by peer (error 104)
2026-08-09T22:58:08Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (2064 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786315935617919-15814-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 22:58:08 dist-test-slave-ncp9 krb5kdc[17296](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 22:58:08 dist-test-slave-ncp9 krb5kdc[17296](info): set up 2 sockets
Aug 09 22:58:08 dist-test-slave-ncp9 krb5kdc[17296](info): commencing operation
krb5kdc: starting...
W20260809 22:58:09.485699 15814 mini_kdc.cc:121] Time spent starting KDC: real 1.174s user 0.010s sys 0.034s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786315935617919-15814-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786315935617919-15814-0/krb5kdc/test-user.keytab.
Aug 09 22:58:09 dist-test-slave-ncp9 krb5kdc[17296](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316289, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T22:58:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T22:58:09Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.113.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.113.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.113.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.190.keytab.
Entry for principal kudu/127.15.113.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.190.keytab.
I20260809 22:58:09.792784 15814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskItYPJR/build/asan/bin/kudu
/tmp/dist-test-taskItYPJR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786315935617919-15814-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786315935617919-15814-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786315935617919-15814-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786315935617919-15814-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.113.190:46867
--webserver_interface=127.15.113.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.190.keytab
--principal=kudu/127.15.113.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.113.148:35199
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.113.190:46867
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786315935617919-15814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786315935617919-15814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786315935617919-15814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:58:10.125856 17314 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:58:10.126137 17314 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:58:10.126271 17314 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:58:10.133280 17314 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 22:58:10.133387 17314 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:58:10.133463 17314 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:58:10.133580 17314 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 22:58:10.133641 17314 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 22:58:10.143159 17314 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.113.148:35199
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786315935617919-15814-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786315935617919-15814-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.113.190:46867
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.190.keytab
--principal=kudu/127.15.113.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.113.190:46867
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786315935617919-15814-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.113.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786315935617919-15814-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:58:10.144359 17314 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:58:10.145699 17314 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:58:10.156908 17324 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:58:10.156845 17321 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:58:10.157334 17320 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:58:10.157532 17314 server_base.cc:1034] running on GCE node
I20260809 22:58:10.158262 17314 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:58:10.159605 17314 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:58:10.160825 17314 hybrid_clock.cc:648] HybridClock initialized: now 1786316290160774 us; error 44 us; skew 500 ppm
I20260809 22:58:10.161181 17314 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:58:10 dist-test-slave-ncp9 krb5kdc[17296](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316290, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.113.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:58:10.164119 17314 init.cc:377] Logged in from keytab as kudu/127.15.113.190@KRBTEST.COM (short username kudu)
I20260809 22:58:10.165491 17314 webserver.cc:460] Webserver started at http://127.15.113.190:38081/ using document root <none> and password file <none>
I20260809 22:58:10.166119 17314 fs_manager.cc:359] Metadata directory not provided
I20260809 22:58:10.166227 17314 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:58:10.166479 17314 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:58:10.169368 17314 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786315935617919-15814-0/minicluster-data/master-0/data/instance:
uuid: "3ccdda189f1744ff86cb3f58028e9b51"
format_stamp: "Formatted at 2026-08-09 22:58:10 on dist-test-slave-ncp9"
server_key: "71ea7cddb7c5d5d6a779f85f23857a82"
server_key_iv: "16e52d29c0f970c827542922fcbed9c0"
server_key_version: "encryptionkey@0"
I20260809 22:58:10.170094 17314 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786315935617919-15814-0/minicluster-data/master-0/wal/instance:
uuid: "3ccdda189f1744ff86cb3f58028e9b51"
format_stamp: "Formatted at 2026-08-09 22:58:10 on dist-test-slave-ncp9"
server_key: "71ea7cddb7c5d5d6a779f85f23857a82"
server_key_iv: "16e52d29c0f970c827542922fcbed9c0"
server_key_version: "encryptionkey@0"
I20260809 22:58:10.175050 17314 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 22:58:10.178359 17340 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:58:10.179702 17314 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 22:58:10.179840 17314 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786315935617919-15814-0/minicluster-data/master-0/data,/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786315935617919-15814-0/minicluster-data/master-0/wal
uuid: "3ccdda189f1744ff86cb3f58028e9b51"
format_stamp: "Formatted at 2026-08-09 22:58:10 on dist-test-slave-ncp9"
server_key: "71ea7cddb7c5d5d6a779f85f23857a82"
server_key_iv: "16e52d29c0f970c827542922fcbed9c0"
server_key_version: "encryptionkey@0"
I20260809 22:58:10.180002 17314 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786315935617919-15814-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786315935617919-15814-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786315935617919-15814-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:58:10.185449 17314 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:58:10.189085 17314 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:58:10.189322 17314 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:58:10.205752 17314 rpc_server.cc:307] RPC server started. Bound to: 127.15.113.190:46867
I20260809 22:58:10.205797 17419 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.113.190:46867 every 8 connection(s)
I20260809 22:58:10.206981 17314 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786315935617919-15814-0/minicluster-data/master-0/data/info.pb
I20260809 22:58:10.210069 15814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskItYPJR/build/asan/bin/kudu as pid 17314
I20260809 22:58:10.210218 15814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786315935617919-15814-0/minicluster-data/master-0/wal/instance
I20260809 22:58:10.210458 17420 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:58:10.210660 15814 external_mini_cluster.cc:1428] Setting key 5bc056f79deffffc8d53d27509af50a8
I20260809 22:58:10.218012 17424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:58:10.212260 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35941 (local address 127.15.113.190:46867)
0809 22:58:10.213065 (+ 805us) server_negotiation.cc:206] Beginning negotiation
0809 22:58:10.213075 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 22:58:10.213119 (+ 44us) server_negotiation.cc:407] Connection header received
0809 22:58:10.214838 (+ 1719us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:58:10.214859 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:58:10.215178 (+ 319us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:58:10.215478 (+ 300us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:58:10.216542 (+ 1064us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:58:10.216549 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:58:10.216565 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 22:58:10.216581 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:58:10.216641 (+ 60us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 22:58:10.216663 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:58:10.216829 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0809 22:58:10.216970 (+ 141us) server_negotiation.cc:299] Negotiation successful
0809 22:58:10.217150 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":542,"thread_start_us":326,"threads_started":1}
I20260809 22:58:10.221465 15814 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 22:58:10.223055 17420 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3ccdda189f1744ff86cb3f58028e9b51: Bootstrap starting.
I20260809 22:58:10.226022 17420 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3ccdda189f1744ff86cb3f58028e9b51: Neither blocks nor log segments found. Creating new log.
I20260809 22:58:10.227026 17420 log.cc:824] T 00000000000000000000000000000000 P 3ccdda189f1744ff86cb3f58028e9b51: Log is configured to *not* fsync() on all Append() calls
I20260809 22:58:10.227867 17424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:58:10.225533 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39295 (local address 127.15.113.190:46867)
0809 22:58:10.225687 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0809 22:58:10.225692 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:58:10.226134 (+ 442us) server_negotiation.cc:407] Connection header received
0809 22:58:10.226289 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:58:10.226294 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:58:10.226342 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:58:10.226429 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:58:10.227281 (+ 852us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:58:10.227286 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:58:10.227290 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 22:58:10.227308 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:58:10.227350 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 22:58:10.227370 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:58:10.227493 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0809 22:58:10.227605 (+ 112us) server_negotiation.cc:299] Negotiation successful
0809 22:58:10.227662 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260809 22:58:10.229951 17420 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3ccdda189f1744ff86cb3f58028e9b51: No bootstrap required, opened a new log
I20260809 22:58:10.237025 17420 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3ccdda189f1744ff86cb3f58028e9b51 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ccdda189f1744ff86cb3f58028e9b51" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 46867 } }
I20260809 22:58:10.237401 17420 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3ccdda189f1744ff86cb3f58028e9b51 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:58:10.237499 17420 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3ccdda189f1744ff86cb3f58028e9b51 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ccdda189f1744ff86cb3f58028e9b51, State: Initialized, Role: FOLLOWER
I20260809 22:58:10.238054 17420 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3ccdda189f1744ff86cb3f58028e9b51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ccdda189f1744ff86cb3f58028e9b51" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 46867 } }
I20260809 22:58:10.238265 17420 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3ccdda189f1744ff86cb3f58028e9b51 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 22:58:10.238394 17420 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3ccdda189f1744ff86cb3f58028e9b51 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 22:58:10.238548 17420 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3ccdda189f1744ff86cb3f58028e9b51 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:58:10.240545 17420 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3ccdda189f1744ff86cb3f58028e9b51 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ccdda189f1744ff86cb3f58028e9b51" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 46867 } }
I20260809 22:58:10.241057 17420 leader_election.cc:302] T 00000000000000000000000000000000 P 3ccdda189f1744ff86cb3f58028e9b51 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3ccdda189f1744ff86cb3f58028e9b51; no voters:
I20260809 22:58:10.241572 17420 leader_election.cc:288] T 00000000000000000000000000000000 P 3ccdda189f1744ff86cb3f58028e9b51 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 22:58:10.241701 17436 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3ccdda189f1744ff86cb3f58028e9b51 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 22:58:10.242324 17436 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3ccdda189f1744ff86cb3f58028e9b51 [term 1 LEADER]: Becoming Leader. State: Replica: 3ccdda189f1744ff86cb3f58028e9b51, State: Running, Role: LEADER
I20260809 22:58:10.242842 17436 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3ccdda189f1744ff86cb3f58028e9b51 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ccdda189f1744ff86cb3f58028e9b51" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 46867 } }
I20260809 22:58:10.243423 17420 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3ccdda189f1744ff86cb3f58028e9b51 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 22:58:10.245627 17438 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3ccdda189f1744ff86cb3f58028e9b51 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3ccdda189f1744ff86cb3f58028e9b51. Latest consensus state: current_term: 1 leader_uuid: "3ccdda189f1744ff86cb3f58028e9b51" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ccdda189f1744ff86cb3f58028e9b51" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 46867 } } }
I20260809 22:58:10.245702 17437 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3ccdda189f1744ff86cb3f58028e9b51 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3ccdda189f1744ff86cb3f58028e9b51" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ccdda189f1744ff86cb3f58028e9b51" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 46867 } } }
I20260809 22:58:10.246050 17438 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3ccdda189f1744ff86cb3f58028e9b51 [sys.catalog]: This master's current role is: LEADER
I20260809 22:58:10.246049 17437 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3ccdda189f1744ff86cb3f58028e9b51 [sys.catalog]: This master's current role is: LEADER
I20260809 22:58:10.249843 17442 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 22:58:10.254931 17442 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 22:58:10.262902 17442 catalog_manager.cc:1347] Generated new cluster ID: 8dc00e43d65243a29ede3c87227fe524
I20260809 22:58:10.262996 17442 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 22:58:10.289753 17442 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 22:58:10.290113 17442 catalog_manager.cc:1504] Loading token signing keys...
I20260809 22:58:10.305956 17442 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3ccdda189f1744ff86cb3f58028e9b51: Generated new TSK 0
I20260809 22:58:10.306681 17442 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 22:58:10.311517 15814 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskItYPJR/build/asan/bin/kudu pid 17314
W20260809 22:58:10.312775 17415 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:35941
W20260809 22:58:10.313407 15814 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskItYPJR/build/asan/bin/kudu pid 17314: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 22:58:10.313530 15814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskItYPJR/build/asan/bin/kudu with pid 17314
2026-08-09T22:58:10Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (2042 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786315935617919-15814-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 22:58:10 dist-test-slave-ncp9 krb5kdc[17465](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 22:58:10 dist-test-slave-ncp9 krb5kdc[17465](info): set up 2 sockets
Aug 09 22:58:10 dist-test-slave-ncp9 krb5kdc[17465](info): commencing operation
krb5kdc: starting...
W20260809 22:58:11.541460 15814 mini_kdc.cc:121] Time spent starting KDC: real 1.187s user 0.004s sys 0.041s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786315935617919-15814-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786315935617919-15814-0/krb5kdc/test-user.keytab.
Aug 09 22:58:11 dist-test-slave-ncp9 krb5kdc[17465](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316291, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T22:58:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T22:58:11Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.113.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.113.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.113.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.190.keytab.
Entry for principal kudu/127.15.113.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.190.keytab.
I20260809 22:58:11.847746 15814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskItYPJR/build/asan/bin/kudu
/tmp/dist-test-taskItYPJR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786315935617919-15814-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786315935617919-15814-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786315935617919-15814-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786315935617919-15814-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.113.190:35759
--webserver_interface=127.15.113.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.190.keytab
--principal=kudu/127.15.113.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.113.148:40569
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.113.190:35759
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786315935617919-15814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786315935617919-15814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786315935617919-15814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:58:12.186028 17483 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:58:12.186307 17483 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:58:12.186399 17483 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:58:12.193544 17483 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 22:58:12.193650 17483 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:58:12.193703 17483 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:58:12.193763 17483 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 22:58:12.193805 17483 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 22:58:12.203493 17483 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.113.148:40569
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786315935617919-15814-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786315935617919-15814-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.113.190:35759
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.190.keytab
--principal=kudu/127.15.113.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.113.190:35759
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786315935617919-15814-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.113.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786315935617919-15814-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:58:12.204732 17483 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:58:12.206068 17483 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:58:12.217453 17490 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:58:12.217499 17491 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:58:12.217521 17495 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:58:12.218756 17483 server_base.cc:1034] running on GCE node
I20260809 22:58:12.219497 17483 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:58:12.220854 17483 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:58:12.222082 17483 hybrid_clock.cc:648] HybridClock initialized: now 1786316292222026 us; error 50 us; skew 500 ppm
I20260809 22:58:12.222416 17483 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:58:12 dist-test-slave-ncp9 krb5kdc[17465](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316292, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.113.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:58:12.229795 17483 init.cc:377] Logged in from keytab as kudu/127.15.113.190@KRBTEST.COM (short username kudu)
I20260809 22:58:12.231197 17483 webserver.cc:460] Webserver started at http://127.15.113.190:44061/ using document root <none> and password file <none>
I20260809 22:58:12.231869 17483 fs_manager.cc:359] Metadata directory not provided
I20260809 22:58:12.231990 17483 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:58:12.232282 17483 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:58:12.235400 17483 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786315935617919-15814-0/minicluster-data/master-0/data/instance:
uuid: "26c195410b34435a99876a2448ffac78"
format_stamp: "Formatted at 2026-08-09 22:58:12 on dist-test-slave-ncp9"
server_key: "bb1d97e319240c596cb5e4e7ffed4825"
server_key_iv: "8067f9c7476ed61671811cc495b6293f"
server_key_version: "encryptionkey@0"
I20260809 22:58:12.236150 17483 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786315935617919-15814-0/minicluster-data/master-0/wal/instance:
uuid: "26c195410b34435a99876a2448ffac78"
format_stamp: "Formatted at 2026-08-09 22:58:12 on dist-test-slave-ncp9"
server_key: "bb1d97e319240c596cb5e4e7ffed4825"
server_key_iv: "8067f9c7476ed61671811cc495b6293f"
server_key_version: "encryptionkey@0"
I20260809 22:58:12.241139 17483 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260809 22:58:12.244403 17503 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:58:12.245680 17483 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 22:58:12.245836 17483 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786315935617919-15814-0/minicluster-data/master-0/data,/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786315935617919-15814-0/minicluster-data/master-0/wal
uuid: "26c195410b34435a99876a2448ffac78"
format_stamp: "Formatted at 2026-08-09 22:58:12 on dist-test-slave-ncp9"
server_key: "bb1d97e319240c596cb5e4e7ffed4825"
server_key_iv: "8067f9c7476ed61671811cc495b6293f"
server_key_version: "encryptionkey@0"
I20260809 22:58:12.246001 17483 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786315935617919-15814-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786315935617919-15814-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786315935617919-15814-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:58:12.260166 17483 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:58:12.263768 17483 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:58:12.264008 17483 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:58:12.280864 17483 rpc_server.cc:307] RPC server started. Bound to: 127.15.113.190:35759
I20260809 22:58:12.280886 17580 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.113.190:35759 every 8 connection(s)
I20260809 22:58:12.282140 17483 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786315935617919-15814-0/minicluster-data/master-0/data/info.pb
I20260809 22:58:12.285570 15814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskItYPJR/build/asan/bin/kudu as pid 17483
I20260809 22:58:12.285755 15814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786315935617919-15814-0/minicluster-data/master-0/wal/instance
I20260809 22:58:12.285943 17581 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:58:12.286337 15814 external_mini_cluster.cc:1428] Setting key 9137bdc9330e2673469fcecdd5c7620f
I20260809 22:58:12.299894 17581 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 26c195410b34435a99876a2448ffac78: Bootstrap starting.
Aug 09 22:58:12 dist-test-slave-ncp9 krb5kdc[17465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316291, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.113.190@KRBTEST.COM
I20260809 22:58:12.303277 17581 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 26c195410b34435a99876a2448ffac78: Neither blocks nor log segments found. Creating new log.
I20260809 22:58:12.304298 17581 log.cc:824] T 00000000000000000000000000000000 P 26c195410b34435a99876a2448ffac78: Log is configured to *not* fsync() on all Append() calls
I20260809 22:58:12.307264 17581 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 26c195410b34435a99876a2448ffac78: No bootstrap required, opened a new log
I20260809 22:58:12.308560 17584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:58:12.288070 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33853 (local address 127.15.113.190:35759)
0809 22:58:12.288851 (+ 781us) server_negotiation.cc:206] Beginning negotiation
0809 22:58:12.288861 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 22:58:12.288932 (+ 71us) server_negotiation.cc:407] Connection header received
0809 22:58:12.290592 (+ 1660us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:58:12.290613 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:58:12.291001 (+ 388us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:58:12.291312 (+ 311us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:58:12.293091 (+ 1779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:58:12.294344 (+ 1253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:58:12.296122 (+ 1778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:58:12.296810 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:58:12.296985 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:58:12.301852 (+ 4867us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:58:12.301859 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:58:12.301873 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:58:12.301913 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:58:12.304823 (+ 2910us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:58:12.305536 (+ 713us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:58:12.305543 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:58:12.305548 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:58:12.305708 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:58:12.306070 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:58:12.306072 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:58:12.306074 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:58:12.306572 (+ 498us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:58:12.306805 (+ 233us) server_negotiation.cc:1035] Waiting for connection context
0809 22:58:12.307232 (+ 427us) server_negotiation.cc:299] Negotiation successful
0809 22:58:12.307580 (+ 348us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":580,"thread_start_us":340,"threads_started":1}
I20260809 22:58:12.312091 15814 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 22:58:12.316345 17581 raft_consensus.cc:348] T 00000000000000000000000000000000 P 26c195410b34435a99876a2448ffac78 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26c195410b34435a99876a2448ffac78" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 35759 } }
I20260809 22:58:12.316798 17581 raft_consensus.cc:374] T 00000000000000000000000000000000 P 26c195410b34435a99876a2448ffac78 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:58:12.316963 17581 raft_consensus.cc:729] T 00000000000000000000000000000000 P 26c195410b34435a99876a2448ffac78 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 26c195410b34435a99876a2448ffac78, State: Initialized, Role: FOLLOWER
I20260809 22:58:12.317616 17581 consensus_queue.cc:260] T 00000000000000000000000000000000 P 26c195410b34435a99876a2448ffac78 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26c195410b34435a99876a2448ffac78" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 35759 } }
I20260809 22:58:12.317920 17581 raft_consensus.cc:388] T 00000000000000000000000000000000 P 26c195410b34435a99876a2448ffac78 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 22:58:12.318073 17581 raft_consensus.cc:482] T 00000000000000000000000000000000 P 26c195410b34435a99876a2448ffac78 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 22:58:12.318255 17581 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 26c195410b34435a99876a2448ffac78 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:58:12.321132 17581 raft_consensus.cc:504] T 00000000000000000000000000000000 P 26c195410b34435a99876a2448ffac78 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26c195410b34435a99876a2448ffac78" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 35759 } }
I20260809 22:58:12.321713 17581 leader_election.cc:302] T 00000000000000000000000000000000 P 26c195410b34435a99876a2448ffac78 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 26c195410b34435a99876a2448ffac78; no voters:
I20260809 22:58:12.322345 17581 leader_election.cc:288] T 00000000000000000000000000000000 P 26c195410b34435a99876a2448ffac78 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 22:58:12.322438 17595 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 26c195410b34435a99876a2448ffac78 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 22:58:12.323513 17595 raft_consensus.cc:686] T 00000000000000000000000000000000 P 26c195410b34435a99876a2448ffac78 [term 1 LEADER]: Becoming Leader. State: Replica: 26c195410b34435a99876a2448ffac78, State: Running, Role: LEADER
I20260809 22:58:12.324040 17595 consensus_queue.cc:237] T 00000000000000000000000000000000 P 26c195410b34435a99876a2448ffac78 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26c195410b34435a99876a2448ffac78" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 35759 } }
I20260809 22:58:12.324589 17581 sys_catalog.cc:564] T 00000000000000000000000000000000 P 26c195410b34435a99876a2448ffac78 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 22:58:12.326771 17598 sys_catalog.cc:455] T 00000000000000000000000000000000 P 26c195410b34435a99876a2448ffac78 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "26c195410b34435a99876a2448ffac78" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26c195410b34435a99876a2448ffac78" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 35759 } } }
I20260809 22:58:12.327126 17598 sys_catalog.cc:458] T 00000000000000000000000000000000 P 26c195410b34435a99876a2448ffac78 [sys.catalog]: This master's current role is: LEADER
I20260809 22:58:12.332196 17599 sys_catalog.cc:455] T 00000000000000000000000000000000 P 26c195410b34435a99876a2448ffac78 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 26c195410b34435a99876a2448ffac78. Latest consensus state: current_term: 1 leader_uuid: "26c195410b34435a99876a2448ffac78" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26c195410b34435a99876a2448ffac78" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 35759 } } }
I20260809 22:58:12.332532 17599 sys_catalog.cc:458] T 00000000000000000000000000000000 P 26c195410b34435a99876a2448ffac78 [sys.catalog]: This master's current role is: LEADER
I20260809 22:58:12.334723 17602 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 22:58:12.334792 17584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:58:12.317406 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44689 (local address 127.15.113.190:35759)
0809 22:58:12.317579 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0809 22:58:12.317586 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:58:12.318089 (+ 503us) server_negotiation.cc:407] Connection header received
0809 22:58:12.318290 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:58:12.318293 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:58:12.318353 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:58:12.318431 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:58:12.320241 (+ 1810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:58:12.320945 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:58:12.322686 (+ 1741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:58:12.323406 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:58:12.323555 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:58:12.326775 (+ 3220us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:58:12.326783 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:58:12.326788 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:58:12.326834 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:58:12.330720 (+ 3886us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:58:12.331545 (+ 825us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:58:12.331551 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:58:12.331555 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:58:12.331701 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:58:12.332228 (+ 527us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:58:12.332234 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:58:12.332238 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:58:12.332706 (+ 468us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:58:12.332989 (+ 283us) server_negotiation.cc:1035] Waiting for connection context
0809 22:58:12.334148 (+ 1159us) server_negotiation.cc:299] Negotiation successful
0809 22:58:12.334386 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260809 22:58:12.340185 17602 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 22:58:12.349290 17602 catalog_manager.cc:1347] Generated new cluster ID: 4cfd79789dd84df78adcc2e854591b4c
I20260809 22:58:12.349381 17602 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 22:58:12.372884 17602 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 22:58:12.374128 17602 catalog_manager.cc:1504] Loading token signing keys...
I20260809 22:58:12.382088 17602 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 26c195410b34435a99876a2448ffac78: Generated new TSK 0
I20260809 22:58:12.382833 17602 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 22:58:12.391722 15814 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskItYPJR/build/asan/bin/kudu pid 17483
I20260809 22:58:12.392748 17576 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:58:12.481408 15814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskItYPJR/build/asan/bin/kudu with pid 17483
2026-08-09T22:58:12Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (2165 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786315935617919-15814-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 22:58:12 dist-test-slave-ncp9 krb5kdc[17624](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 22:58:12 dist-test-slave-ncp9 krb5kdc[17624](info): set up 2 sockets
Aug 09 22:58:12 dist-test-slave-ncp9 krb5kdc[17624](info): commencing operation
krb5kdc: starting...
W20260809 22:58:13.707091 15814 mini_kdc.cc:121] Time spent starting KDC: real 1.187s user 0.000s sys 0.049s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786315935617919-15814-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786315935617919-15814-0/krb5kdc/test-user.keytab.
Aug 09 22:58:13 dist-test-slave-ncp9 krb5kdc[17624](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316293, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T22:58:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T22:58:13Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.113.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.113.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.113.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.190.keytab.
Entry for principal kudu/127.15.113.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.190.keytab.
I20260809 22:58:14.035635 15814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskItYPJR/build/asan/bin/kudu
/tmp/dist-test-taskItYPJR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786315935617919-15814-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786315935617919-15814-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786315935617919-15814-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786315935617919-15814-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.113.190:36959
--webserver_interface=127.15.113.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.190.keytab
--principal=kudu/127.15.113.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.113.148:35905
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.113.190:36959
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786315935617919-15814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786315935617919-15814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786315935617919-15814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:58:14.382534 17651 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:58:14.382817 17651 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:58:14.382961 17651 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:58:14.390185 17651 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 22:58:14.390303 17651 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:58:14.390383 17651 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:58:14.390462 17651 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 22:58:14.390522 17651 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 22:58:14.400027 17651 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.113.148:35905
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786315935617919-15814-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786315935617919-15814-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.113.190:36959
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.190.keytab
--principal=kudu/127.15.113.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.113.190:36959
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786315935617919-15814-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.113.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786315935617919-15814-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:58:14.401237 17651 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:58:14.402549 17651 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:58:14.413554 17657 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:58:14.413650 17658 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:58:14.413621 17660 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:58:14.414405 17651 server_base.cc:1034] running on GCE node
I20260809 22:58:14.415071 17651 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:58:14.416282 17651 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:58:14.417459 17651 hybrid_clock.cc:648] HybridClock initialized: now 1786316294417411 us; error 40 us; skew 500 ppm
I20260809 22:58:14.417778 17651 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:58:14 dist-test-slave-ncp9 krb5kdc[17624](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316294, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.113.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:58:14.420680 17651 init.cc:377] Logged in from keytab as kudu/127.15.113.190@KRBTEST.COM (short username kudu)
I20260809 22:58:14.422079 17651 webserver.cc:460] Webserver started at http://127.15.113.190:44787/ using document root <none> and password file <none>
I20260809 22:58:14.422698 17651 fs_manager.cc:359] Metadata directory not provided
I20260809 22:58:14.422799 17651 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:58:14.423034 17651 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:58:14.425868 17651 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786315935617919-15814-0/minicluster-data/master-0/data/instance:
uuid: "3a8666dbaddb4903b87e59db70ef64a4"
format_stamp: "Formatted at 2026-08-09 22:58:14 on dist-test-slave-ncp9"
server_key: "233977a8db3379a829f07562d02550fe"
server_key_iv: "e3cc107cc69044ef629d3822b38c5ae7"
server_key_version: "encryptionkey@0"
I20260809 22:58:14.426549 17651 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786315935617919-15814-0/minicluster-data/master-0/wal/instance:
uuid: "3a8666dbaddb4903b87e59db70ef64a4"
format_stamp: "Formatted at 2026-08-09 22:58:14 on dist-test-slave-ncp9"
server_key: "233977a8db3379a829f07562d02550fe"
server_key_iv: "e3cc107cc69044ef629d3822b38c5ae7"
server_key_version: "encryptionkey@0"
I20260809 22:58:14.431313 17651 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260809 22:58:14.434635 17671 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:58:14.435827 17651 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 22:58:14.435974 17651 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786315935617919-15814-0/minicluster-data/master-0/data,/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786315935617919-15814-0/minicluster-data/master-0/wal
uuid: "3a8666dbaddb4903b87e59db70ef64a4"
format_stamp: "Formatted at 2026-08-09 22:58:14 on dist-test-slave-ncp9"
server_key: "233977a8db3379a829f07562d02550fe"
server_key_iv: "e3cc107cc69044ef629d3822b38c5ae7"
server_key_version: "encryptionkey@0"
I20260809 22:58:14.436133 17651 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786315935617919-15814-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786315935617919-15814-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786315935617919-15814-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:58:14.469216 17651 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:58:14.472846 17651 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:58:14.473107 17651 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:58:14.490006 17651 rpc_server.cc:307] RPC server started. Bound to: 127.15.113.190:36959
I20260809 22:58:14.490036 17753 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.113.190:36959 every 8 connection(s)
I20260809 22:58:14.491273 17651 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786315935617919-15814-0/minicluster-data/master-0/data/info.pb
I20260809 22:58:14.494846 17756 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:58:14.495338 15814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskItYPJR/build/asan/bin/kudu as pid 17651
I20260809 22:58:14.495497 15814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786315935617919-15814-0/minicluster-data/master-0/wal/instance
I20260809 22:58:14.495931 15814 external_mini_cluster.cc:1428] Setting key 09135d82f119538203da5f48fa0f7ad4
I20260809 22:58:14.507654 17756 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3a8666dbaddb4903b87e59db70ef64a4: Bootstrap starting.
I20260809 22:58:14.509115 17761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:58:14.497564 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40287 (local address 127.15.113.190:36959)
0809 22:58:14.498332 (+ 768us) server_negotiation.cc:206] Beginning negotiation
0809 22:58:14.498343 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 22:58:14.498383 (+ 40us) server_negotiation.cc:407] Connection header received
0809 22:58:14.500510 (+ 2127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:58:14.500531 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:58:14.500929 (+ 398us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:58:14.501290 (+ 361us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:58:14.501857 (+ 567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:58:14.503085 (+ 1228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:58:14.504954 (+ 1869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:58:14.505630 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:58:14.505750 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:58:14.506939 (+ 1189us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:58:14.506946 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:58:14.506959 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 22:58:14.507458 (+ 499us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:58:14.507530 (+ 72us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 22:58:14.507553 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:58:14.507632 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0809 22:58:14.507895 (+ 263us) server_negotiation.cc:299] Negotiation successful
0809 22:58:14.508125 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":584,"thread_start_us":332,"threads_started":1}
I20260809 22:58:14.510819 17756 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3a8666dbaddb4903b87e59db70ef64a4: Neither blocks nor log segments found. Creating new log.
I20260809 22:58:14.511988 17756 log.cc:824] T 00000000000000000000000000000000 P 3a8666dbaddb4903b87e59db70ef64a4: Log is configured to *not* fsync() on all Append() calls
I20260809 22:58:14.512346 15814 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 22:58:14.514575 17756 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3a8666dbaddb4903b87e59db70ef64a4: No bootstrap required, opened a new log
I20260809 22:58:14.522706 17761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:58:14.516485 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56753 (local address 127.15.113.190:36959)
0809 22:58:14.516674 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0809 22:58:14.516680 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:58:14.517090 (+ 410us) server_negotiation.cc:407] Connection header received
0809 22:58:14.517243 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:58:14.517248 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:58:14.517326 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:58:14.517421 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:58:14.517864 (+ 443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:58:14.518941 (+ 1077us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:58:14.520215 (+ 1274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:58:14.520944 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:58:14.521059 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:58:14.521988 (+ 929us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:58:14.521994 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:58:14.521999 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 22:58:14.522020 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:58:14.522066 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 22:58:14.522085 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:58:14.522177 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0809 22:58:14.522290 (+ 113us) server_negotiation.cc:299] Negotiation successful
0809 22:58:14.522380 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260809 22:58:14.523531 17756 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3a8666dbaddb4903b87e59db70ef64a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3a8666dbaddb4903b87e59db70ef64a4" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 36959 } }
I20260809 22:58:14.523943 17756 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3a8666dbaddb4903b87e59db70ef64a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:58:14.524039 17756 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3a8666dbaddb4903b87e59db70ef64a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a8666dbaddb4903b87e59db70ef64a4, State: Initialized, Role: FOLLOWER
I20260809 22:58:14.524593 17756 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3a8666dbaddb4903b87e59db70ef64a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3a8666dbaddb4903b87e59db70ef64a4" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 36959 } }
I20260809 22:58:14.524821 17756 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3a8666dbaddb4903b87e59db70ef64a4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 22:58:14.524976 17756 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3a8666dbaddb4903b87e59db70ef64a4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 22:58:14.525146 17756 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3a8666dbaddb4903b87e59db70ef64a4 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:58:14.527490 17756 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3a8666dbaddb4903b87e59db70ef64a4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3a8666dbaddb4903b87e59db70ef64a4" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 36959 } }
I20260809 22:58:14.527973 17756 leader_election.cc:302] T 00000000000000000000000000000000 P 3a8666dbaddb4903b87e59db70ef64a4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3a8666dbaddb4903b87e59db70ef64a4; no voters:
I20260809 22:58:14.528512 17756 leader_election.cc:288] T 00000000000000000000000000000000 P 3a8666dbaddb4903b87e59db70ef64a4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 22:58:14.528637 17770 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3a8666dbaddb4903b87e59db70ef64a4 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 22:58:14.529345 17770 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3a8666dbaddb4903b87e59db70ef64a4 [term 1 LEADER]: Becoming Leader. State: Replica: 3a8666dbaddb4903b87e59db70ef64a4, State: Running, Role: LEADER
I20260809 22:58:14.529832 17770 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3a8666dbaddb4903b87e59db70ef64a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3a8666dbaddb4903b87e59db70ef64a4" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 36959 } }
I20260809 22:58:14.530591 17756 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3a8666dbaddb4903b87e59db70ef64a4 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 22:58:14.532493 17772 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3a8666dbaddb4903b87e59db70ef64a4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3a8666dbaddb4903b87e59db70ef64a4. Latest consensus state: current_term: 1 leader_uuid: "3a8666dbaddb4903b87e59db70ef64a4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3a8666dbaddb4903b87e59db70ef64a4" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 36959 } } }
I20260809 22:58:14.532804 17772 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3a8666dbaddb4903b87e59db70ef64a4 [sys.catalog]: This master's current role is: LEADER
I20260809 22:58:14.533072 17771 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3a8666dbaddb4903b87e59db70ef64a4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3a8666dbaddb4903b87e59db70ef64a4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3a8666dbaddb4903b87e59db70ef64a4" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 36959 } } }
I20260809 22:58:14.533381 17771 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3a8666dbaddb4903b87e59db70ef64a4 [sys.catalog]: This master's current role is: LEADER
I20260809 22:58:14.537528 17780 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 22:58:14.542346 17780 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 22:58:14.551262 17780 catalog_manager.cc:1347] Generated new cluster ID: f4381a1973ce4f25987cad14d30ce389
I20260809 22:58:14.551403 17780 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 22:58:14.568969 17780 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 22:58:14.570185 17780 catalog_manager.cc:1504] Loading token signing keys...
I20260809 22:58:14.575773 17780 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3a8666dbaddb4903b87e59db70ef64a4: Generated new TSK 0
I20260809 22:58:14.576488 17780 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 22:58:14.589121 15814 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskItYPJR/build/asan/bin/kudu pid 17651
W20260809 22:58:14.590179 17748 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:40287
W20260809 22:58:14.590854 15814 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskItYPJR/build/asan/bin/kudu pid 17651: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 22:58:14.590986 15814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskItYPJR/build/asan/bin/kudu with pid 17651
2026-08-09T22:58:14Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (2114 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786315935617919-15814-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 22:58:14 dist-test-slave-ncp9 krb5kdc[17799](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 22:58:14 dist-test-slave-ncp9 krb5kdc[17799](info): set up 2 sockets
Aug 09 22:58:14 dist-test-slave-ncp9 krb5kdc[17799](info): commencing operation
krb5kdc: starting...
W20260809 22:58:15.816931 15814 mini_kdc.cc:121] Time spent starting KDC: real 1.183s user 0.000s sys 0.049s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786315935617919-15814-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786315935617919-15814-0/krb5kdc/test-user.keytab.
Aug 09 22:58:16 dist-test-slave-ncp9 krb5kdc[17799](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316296, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T22:58:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T22:58:16Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.113.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.113.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.113.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.190.keytab.
Entry for principal kudu/127.15.113.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.190.keytab.
I20260809 22:58:16.120080 15814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskItYPJR/build/asan/bin/kudu
/tmp/dist-test-taskItYPJR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786315935617919-15814-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786315935617919-15814-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786315935617919-15814-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786315935617919-15814-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.113.190:40589
--webserver_interface=127.15.113.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.190.keytab
--principal=kudu/127.15.113.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.113.148:46797
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.113.190:40589
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786315935617919-15814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786315935617919-15814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786315935617919-15814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:58:16.450641 17818 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:58:16.450922 17818 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:58:16.451025 17818 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:58:16.458030 17818 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 22:58:16.458137 17818 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:58:16.458189 17818 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:58:16.458249 17818 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 22:58:16.458293 17818 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 22:58:16.467841 17818 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.113.148:46797
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786315935617919-15814-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786315935617919-15814-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.113.190:40589
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.190.keytab
--principal=kudu/127.15.113.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.113.190:40589
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786315935617919-15814-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.113.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786315935617919-15814-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:58:16.469092 17818 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:58:16.470343 17818 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:58:16.481684 17824 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:58:16.481751 17831 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:58:16.481784 17827 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:58:16.482838 17818 server_base.cc:1034] running on GCE node
I20260809 22:58:16.483556 17818 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:58:16.484978 17818 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:58:16.486184 17818 hybrid_clock.cc:648] HybridClock initialized: now 1786316296486135 us; error 41 us; skew 500 ppm
I20260809 22:58:16.486509 17818 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:58:16 dist-test-slave-ncp9 krb5kdc[17799](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316296, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.113.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:58:16.489496 17818 init.cc:377] Logged in from keytab as kudu/127.15.113.190@KRBTEST.COM (short username kudu)
I20260809 22:58:16.490990 17818 webserver.cc:460] Webserver started at http://127.15.113.190:36199/ using document root <none> and password file <none>
I20260809 22:58:16.491667 17818 fs_manager.cc:359] Metadata directory not provided
I20260809 22:58:16.491786 17818 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:58:16.492143 17818 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:58:16.495193 17818 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786315935617919-15814-0/minicluster-data/master-0/data/instance:
uuid: "c5e6ce4083724a4db8452cbf339ad497"
format_stamp: "Formatted at 2026-08-09 22:58:16 on dist-test-slave-ncp9"
server_key: "a01b380ad3f80aaa9e7d8117cf64c0d6"
server_key_iv: "2a79ee93d1f3bb03ac091b705f8343fc"
server_key_version: "encryptionkey@0"
I20260809 22:58:16.495947 17818 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786315935617919-15814-0/minicluster-data/master-0/wal/instance:
uuid: "c5e6ce4083724a4db8452cbf339ad497"
format_stamp: "Formatted at 2026-08-09 22:58:16 on dist-test-slave-ncp9"
server_key: "a01b380ad3f80aaa9e7d8117cf64c0d6"
server_key_iv: "2a79ee93d1f3bb03ac091b705f8343fc"
server_key_version: "encryptionkey@0"
I20260809 22:58:16.500959 17818 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260809 22:58:16.504352 17843 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:58:16.505673 17818 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 22:58:16.505836 17818 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786315935617919-15814-0/minicluster-data/master-0/data,/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786315935617919-15814-0/minicluster-data/master-0/wal
uuid: "c5e6ce4083724a4db8452cbf339ad497"
format_stamp: "Formatted at 2026-08-09 22:58:16 on dist-test-slave-ncp9"
server_key: "a01b380ad3f80aaa9e7d8117cf64c0d6"
server_key_iv: "2a79ee93d1f3bb03ac091b705f8343fc"
server_key_version: "encryptionkey@0"
I20260809 22:58:16.506014 17818 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786315935617919-15814-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786315935617919-15814-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786315935617919-15814-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:58:16.511487 17818 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:58:16.515236 17818 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:58:16.515506 17818 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:58:16.532987 17921 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.113.190:40589 every 8 connection(s)
I20260809 22:58:16.533052 17818 rpc_server.cc:307] RPC server started. Bound to: 127.15.113.190:40589
I20260809 22:58:16.534456 17818 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786315935617919-15814-0/minicluster-data/master-0/data/info.pb
I20260809 22:58:16.536021 15814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskItYPJR/build/asan/bin/kudu as pid 17818
I20260809 22:58:16.536187 15814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786315935617919-15814-0/minicluster-data/master-0/wal/instance
I20260809 22:58:16.536623 15814 external_mini_cluster.cc:1428] Setting key 8a311220f9d22080b457ab3de54eeafc
I20260809 22:58:16.538877 17925 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:58:16.548007 17935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:58:16.538469 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49467 (local address 127.15.113.190:40589)
0809 22:58:16.539346 (+ 877us) server_negotiation.cc:206] Beginning negotiation
0809 22:58:16.539362 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0809 22:58:16.539413 (+ 51us) server_negotiation.cc:407] Connection header received
0809 22:58:16.543988 (+ 4575us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:58:16.544025 (+ 37us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:58:16.544509 (+ 484us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:58:16.544986 (+ 477us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:58:16.546008 (+ 1022us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:58:16.546016 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:58:16.546035 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 22:58:16.546057 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:58:16.546146 (+ 89us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 22:58:16.546176 (+ 30us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:58:16.546410 (+ 234us) server_negotiation.cc:1035] Waiting for connection context
0809 22:58:16.546569 (+ 159us) server_negotiation.cc:299] Negotiation successful
0809 22:58:16.546770 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":610,"thread_start_us":362,"threads_started":1}
I20260809 22:58:16.551638 15814 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 22:58:16.555621 17925 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c5e6ce4083724a4db8452cbf339ad497: Bootstrap starting.
I20260809 22:58:16.559338 17935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:58:16.556846 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41715 (local address 127.15.113.190:40589)
0809 22:58:16.557065 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0809 22:58:16.557072 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:58:16.557502 (+ 430us) server_negotiation.cc:407] Connection header received
0809 22:58:16.557673 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:58:16.557677 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:58:16.557725 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:58:16.557800 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:58:16.558754 (+ 954us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:58:16.558758 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:58:16.558761 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 22:58:16.558775 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:58:16.558811 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 22:58:16.558825 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:58:16.558901 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0809 22:58:16.559033 (+ 132us) server_negotiation.cc:299] Negotiation successful
0809 22:58:16.559076 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260809 22:58:16.559556 17925 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c5e6ce4083724a4db8452cbf339ad497: Neither blocks nor log segments found. Creating new log.
I20260809 22:58:16.560737 17925 log.cc:824] T 00000000000000000000000000000000 P c5e6ce4083724a4db8452cbf339ad497: Log is configured to *not* fsync() on all Append() calls
I20260809 22:58:16.563352 17925 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c5e6ce4083724a4db8452cbf339ad497: No bootstrap required, opened a new log
I20260809 22:58:16.570899 17925 raft_consensus.cc:348] T 00000000000000000000000000000000 P c5e6ce4083724a4db8452cbf339ad497 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5e6ce4083724a4db8452cbf339ad497" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 40589 } }
I20260809 22:58:16.571250 17925 raft_consensus.cc:374] T 00000000000000000000000000000000 P c5e6ce4083724a4db8452cbf339ad497 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:58:16.571324 17925 raft_consensus.cc:729] T 00000000000000000000000000000000 P c5e6ce4083724a4db8452cbf339ad497 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c5e6ce4083724a4db8452cbf339ad497, State: Initialized, Role: FOLLOWER
I20260809 22:58:16.571844 17925 consensus_queue.cc:260] T 00000000000000000000000000000000 P c5e6ce4083724a4db8452cbf339ad497 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5e6ce4083724a4db8452cbf339ad497" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 40589 } }
I20260809 22:58:16.572045 17925 raft_consensus.cc:388] T 00000000000000000000000000000000 P c5e6ce4083724a4db8452cbf339ad497 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 22:58:16.572150 17925 raft_consensus.cc:482] T 00000000000000000000000000000000 P c5e6ce4083724a4db8452cbf339ad497 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 22:58:16.572270 17925 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c5e6ce4083724a4db8452cbf339ad497 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:58:16.574472 17925 raft_consensus.cc:504] T 00000000000000000000000000000000 P c5e6ce4083724a4db8452cbf339ad497 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5e6ce4083724a4db8452cbf339ad497" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 40589 } }
I20260809 22:58:16.574985 17925 leader_election.cc:302] T 00000000000000000000000000000000 P c5e6ce4083724a4db8452cbf339ad497 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c5e6ce4083724a4db8452cbf339ad497; no voters:
I20260809 22:58:16.575533 17925 leader_election.cc:288] T 00000000000000000000000000000000 P c5e6ce4083724a4db8452cbf339ad497 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 22:58:16.575635 17942 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c5e6ce4083724a4db8452cbf339ad497 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 22:58:16.576246 17942 raft_consensus.cc:686] T 00000000000000000000000000000000 P c5e6ce4083724a4db8452cbf339ad497 [term 1 LEADER]: Becoming Leader. State: Replica: c5e6ce4083724a4db8452cbf339ad497, State: Running, Role: LEADER
I20260809 22:58:16.576756 17942 consensus_queue.cc:237] T 00000000000000000000000000000000 P c5e6ce4083724a4db8452cbf339ad497 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5e6ce4083724a4db8452cbf339ad497" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 40589 } }
I20260809 22:58:16.577545 17925 sys_catalog.cc:564] T 00000000000000000000000000000000 P c5e6ce4083724a4db8452cbf339ad497 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 22:58:16.579617 17943 sys_catalog.cc:455] T 00000000000000000000000000000000 P c5e6ce4083724a4db8452cbf339ad497 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c5e6ce4083724a4db8452cbf339ad497" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5e6ce4083724a4db8452cbf339ad497" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 40589 } } }
I20260809 22:58:16.580067 17943 sys_catalog.cc:458] T 00000000000000000000000000000000 P c5e6ce4083724a4db8452cbf339ad497 [sys.catalog]: This master's current role is: LEADER
I20260809 22:58:16.580925 17954 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 22:58:16.581626 17945 sys_catalog.cc:455] T 00000000000000000000000000000000 P c5e6ce4083724a4db8452cbf339ad497 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c5e6ce4083724a4db8452cbf339ad497. Latest consensus state: current_term: 1 leader_uuid: "c5e6ce4083724a4db8452cbf339ad497" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5e6ce4083724a4db8452cbf339ad497" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 40589 } } }
I20260809 22:58:16.581962 17945 sys_catalog.cc:458] T 00000000000000000000000000000000 P c5e6ce4083724a4db8452cbf339ad497 [sys.catalog]: This master's current role is: LEADER
I20260809 22:58:16.586382 17954 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 22:58:16.596112 17954 catalog_manager.cc:1347] Generated new cluster ID: e0d1b7ceea33462ea5ecde917408dc4e
I20260809 22:58:16.596216 17954 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 22:58:16.605629 17954 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 22:58:16.605988 17954 catalog_manager.cc:1504] Loading token signing keys...
I20260809 22:58:16.618031 17954 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c5e6ce4083724a4db8452cbf339ad497: Generated new TSK 0
I20260809 22:58:16.618791 17954 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 22:58:16.625113 15814 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskItYPJR/build/asan/bin/kudu pid 17818
W20260809 22:58:16.626302 17915 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:49467
W20260809 22:58:16.627012 15814 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskItYPJR/build/asan/bin/kudu pid 17818: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 22:58:16.627166 15814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskItYPJR/build/asan/bin/kudu with pid 17818
2026-08-09T22:58:16Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (2033 ms)
[----------] 18 tests from AuthTokenIssuingTest (38262 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786315935617919-15814-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 22:58:16 dist-test-slave-ncp9 krb5kdc[17974](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 22:58:16 dist-test-slave-ncp9 krb5kdc[17974](info): set up 2 sockets
Aug 09 22:58:16 dist-test-slave-ncp9 krb5kdc[17974](info): commencing operation
krb5kdc: starting...
W20260809 22:58:17.858422 15814 mini_kdc.cc:121] Time spent starting KDC: real 1.190s user 0.004s sys 0.048s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786315935617919-15814-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786315935617919-15814-0/krb5kdc/test-user.keytab.
Aug 09 22:58:18 dist-test-slave-ncp9 krb5kdc[17974](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316298, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T22:58:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T22:58:18Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.113.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.113.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.113.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.190.keytab.
Entry for principal kudu/127.15.113.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.190.keytab.
I20260809 22:58:18.166308 15814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskItYPJR/build/asan/bin/kudu
/tmp/dist-test-taskItYPJR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786315935617919-15814-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786315935617919-15814-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786315935617919-15814-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786315935617919-15814-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.113.190:38393
--webserver_interface=127.15.113.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.190.keytab
--principal=kudu/127.15.113.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.113.148:43367
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.113.190:38393,127.15.113.189:35697,127.15.113.188:45579
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786315935617919-15814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786315935617919-15814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786315935617919-15814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:58:18.495237 17991 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260809 22:58:18.495553 17991 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:58:18.495622 17991 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:58:18.495702 17991 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:58:18.502698 17991 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 22:58:18.502784 17991 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:58:18.502838 17991 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:58:18.502892 17991 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 22:58:18.502951 17991 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 22:58:18.512248 17991 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.113.148:43367
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786315935617919-15814-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786315935617919-15814-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.113.190:38393,127.15.113.189:35697,127.15.113.188:45579
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.190.keytab
--principal=kudu/127.15.113.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.113.190:38393
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786315935617919-15814-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.113.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786315935617919-15814-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:58:18.513432 17991 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:58:18.514670 17991 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:58:18.526108 18003 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:58:18.526229 18006 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:58:18.526285 17996 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:58:18.527138 17991 server_base.cc:1034] running on GCE node
I20260809 22:58:18.527794 17991 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:58:18.529057 17991 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:58:18.530265 17991 hybrid_clock.cc:648] HybridClock initialized: now 1786316298530205 us; error 33 us; skew 500 ppm
I20260809 22:58:18.530560 17991 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:58:18 dist-test-slave-ncp9 krb5kdc[17974](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316298, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.113.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:58:18.537015 17991 init.cc:377] Logged in from keytab as kudu/127.15.113.190@KRBTEST.COM (short username kudu)
I20260809 22:58:18.538376 17991 webserver.cc:460] Webserver started at http://127.15.113.190:33343/ using document root <none> and password file <none>
I20260809 22:58:18.539021 17991 fs_manager.cc:359] Metadata directory not provided
I20260809 22:58:18.539121 17991 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:58:18.539433 17991 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:58:18.542336 17991 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786315935617919-15814-0/minicluster-data/master-0/data/instance:
uuid: "cf333d32995a493bb7cef3ede9926ce2"
format_stamp: "Formatted at 2026-08-09 22:58:18 on dist-test-slave-ncp9"
server_key: "85840e20293ddb92fb79b8f43e6c4aea"
server_key_iv: "0dcd58cbb4e356d333089411218f3b00"
server_key_version: "encryptionkey@0"
I20260809 22:58:18.543088 17991 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786315935617919-15814-0/minicluster-data/master-0/wal/instance:
uuid: "cf333d32995a493bb7cef3ede9926ce2"
format_stamp: "Formatted at 2026-08-09 22:58:18 on dist-test-slave-ncp9"
server_key: "85840e20293ddb92fb79b8f43e6c4aea"
server_key_iv: "0dcd58cbb4e356d333089411218f3b00"
server_key_version: "encryptionkey@0"
I20260809 22:58:18.547920 17991 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.005s
I20260809 22:58:18.551209 18022 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:58:18.552480 17991 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260809 22:58:18.552636 17991 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786315935617919-15814-0/minicluster-data/master-0/data,/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786315935617919-15814-0/minicluster-data/master-0/wal
uuid: "cf333d32995a493bb7cef3ede9926ce2"
format_stamp: "Formatted at 2026-08-09 22:58:18 on dist-test-slave-ncp9"
server_key: "85840e20293ddb92fb79b8f43e6c4aea"
server_key_iv: "0dcd58cbb4e356d333089411218f3b00"
server_key_version: "encryptionkey@0"
I20260809 22:58:18.552827 17991 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786315935617919-15814-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786315935617919-15814-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786315935617919-15814-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:58:18.574112 17991 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:58:18.577766 17991 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:58:18.578001 17991 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:58:18.594961 17991 rpc_server.cc:307] RPC server started. Bound to: 127.15.113.190:38393
I20260809 22:58:18.594992 18095 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.113.190:38393 every 8 connection(s)
I20260809 22:58:18.596271 17991 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786315935617919-15814-0/minicluster-data/master-0/data/info.pb
I20260809 22:58:18.599859 18096 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:58:18.603554 15814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskItYPJR/build/asan/bin/kudu as pid 17991
I20260809 22:58:18.603799 15814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786315935617919-15814-0/minicluster-data/master-0/wal/instance
I20260809 22:58:18.604616 15814 external_mini_cluster.cc:1428] Setting key afae240a0317f1b8d15392de144660c0
I20260809 22:58:18.607631 18096 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.113.190" port: 38393 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 22:58:18 dist-test-slave-ncp9 krb5kdc[17974](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316298, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.113.190@KRBTEST.COM for kudu/127.15.113.190@KRBTEST.COM
I20260809 22:58:18.630152 18099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:58:18.611152 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58952 (local address 127.15.113.190:38393)
0809 22:58:18.612849 (+ 1697us) server_negotiation.cc:206] Beginning negotiation
0809 22:58:18.612855 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:58:18.612922 (+ 67us) server_negotiation.cc:407] Connection header received
0809 22:58:18.614691 (+ 1769us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:58:18.614696 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:58:18.614899 (+ 203us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:58:18.615053 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:58:18.616531 (+ 1478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:58:18.617756 (+ 1225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:58:18.619523 (+ 1767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:58:18.620041 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:58:18.620170 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:58:18.623606 (+ 3436us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:58:18.623613 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:58:18.623634 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:58:18.623666 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:58:18.626264 (+ 2598us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:58:18.627099 (+ 835us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:58:18.627105 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:58:18.627107 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:58:18.627257 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:58:18.627645 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:58:18.627648 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:58:18.627650 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:58:18.628098 (+ 448us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:58:18.628248 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0809 22:58:18.628828 (+ 580us) server_negotiation.cc:299] Negotiation successful
0809 22:58:18.629110 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1505,"spinlock_wait_cycles":21888,"thread_start_us":1266,"threads_started":1}
I20260809 22:58:18.630190 18098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:58:18.611157 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:38393 (local address 127.0.0.1:58952)
0809 22:58:18.612022 (+ 865us) negotiation.cc:107] Waiting for socket to connect
0809 22:58:18.612080 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0809 22:58:18.614341 (+ 2261us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:58:18.615130 (+ 789us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:58:18.615137 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:58:18.615859 (+ 722us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:58:18.616318 (+ 459us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:58:18.616341 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:58:18.617902 (+ 1561us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:58:18.617907 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:58:18.619384 (+ 1477us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:58:18.619399 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:58:18.620266 (+ 867us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:58:18.620276 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:58:18.620438 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:58:18.621092 (+ 654us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:58:18.621118 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:58:18.623387 (+ 2269us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:58:18.626386 (+ 2999us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:58:18.626398 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:58:18.626408 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:58:18.626977 (+ 569us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:58:18.627361 (+ 384us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:58:18.627365 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:58:18.627368 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:58:18.627554 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:58:18.628241 (+ 687us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:58:18.628248 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:58:18.628466 (+ 218us) client_negotiation.cc:769] Sending connection context
0809 22:58:18.628717 (+ 251us) client_negotiation.cc:240] Negotiation successful
0809 22:58:18.629297 (+ 580us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":600,"mutex_wait_us":95,"thread_start_us":364,"threads_started":1}
I20260809 22:58:18.633117 18096 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.113.189" port: 35697 } has no permanent_uuid. Determining permanent_uuid...
I20260809 22:58:18.634099 18098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:58:18.633689 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.189:35697 (local address 127.0.0.1:60716)
0809 22:58:18.633862 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0809 22:58:18.633981 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.113.189:35697: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260809 22:58:18.634359 18026 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.15.113.189:35697: connect: Connection refused (error 111)
I20260809 22:58:18.635883 18098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:58:18.635472 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.189:35697 (local address 127.0.0.1:60720)
0809 22:58:18.635641 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0809 22:58:18.635727 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.113.189:35697: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260809 22:58:18.636098 18096 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.113.189:35697: Network error: Client connection negotiation failed: client connection to 127.15.113.189:35697: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.15.113.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.113.189@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.113.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.189.keytab.
Entry for principal kudu/127.15.113.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.189.keytab.
I20260809 22:58:18.675698 15814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskItYPJR/build/asan/bin/kudu
/tmp/dist-test-taskItYPJR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786315935617919-15814-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786315935617919-15814-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786315935617919-15814-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786315935617919-15814-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.113.189:35697
--webserver_interface=127.15.113.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.189.keytab
--principal=kudu/127.15.113.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.113.148:43367
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.113.190:38393,127.15.113.189:35697,127.15.113.188:45579
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786315935617919-15814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786315935617919-15814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786315935617919-15814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 22:58:18.685397 18096 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.113.189" port: 35697 } attempt: 1
I20260809 22:58:18.686497 18098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:58:18.686042 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.189:35697 (local address 127.0.0.1:55018)
0809 22:58:18.686253 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0809 22:58:18.686374 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.113.189:35697: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
I20260809 22:58:18.687507 18098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:58:18.687134 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.189:35697 (local address 127.0.0.1:55028)
0809 22:58:18.687284 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0809 22:58:18.687353 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.113.189:35697: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260809 22:58:18.687681 18096 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.113.189:35697: Network error: Client connection negotiation failed: client connection to 127.15.113.189:35697: connect: Connection refused (error 111)
I20260809 22:58:18.755899 18096 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.113.189" port: 35697 } attempt: 2
I20260809 22:58:18.757031 18098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:58:18.756528 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.189:35697 (local address 127.0.0.1:55038)
0809 22:58:18.756759 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0809 22:58:18.756849 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.113.189:35697: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260809 22:58:18.758111 18098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:58:18.757753 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.189:35697 (local address 127.0.0.1:55046)
0809 22:58:18.757911 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0809 22:58:18.757980 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.113.189:35697: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260809 22:58:18.758294 18096 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.113.189:35697: Network error: Client connection negotiation failed: client connection to 127.15.113.189:35697: connect: Connection refused (error 111)
I20260809 22:58:18.849527 18096 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.113.189" port: 35697 } attempt: 3
I20260809 22:58:18.850736 18098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:58:18.850311 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.189:35697 (local address 127.0.0.1:55048)
0809 22:58:18.850511 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0809 22:58:18.850602 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.113.189:35697: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
I20260809 22:58:18.852272 18098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:58:18.851825 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.189:35697 (local address 127.0.0.1:55062)
0809 22:58:18.852008 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0809 22:58:18.852097 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.113.189:35697: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260809 22:58:18.852722 18096 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.113.189:35697: Network error: Client connection negotiation failed: client connection to 127.15.113.189:35697: connect: Connection refused (error 111)
I20260809 22:58:18.996061 18096 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.113.189" port: 35697 } attempt: 4
I20260809 22:58:18.997215 18098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:58:18.996683 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.189:35697 (local address 127.0.0.1:55066)
0809 22:58:18.997002 (+ 319us) negotiation.cc:107] Waiting for socket to connect
0809 22:58:18.997087 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.113.189:35697: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":145}
I20260809 22:58:18.998204 18098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:58:18.997869 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.189:35697 (local address 127.0.0.1:55076)
0809 22:58:18.998010 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0809 22:58:18.998087 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.113.189:35697: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260809 22:58:18.998384 18096 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.113.189:35697: Network error: Client connection negotiation failed: client connection to 127.15.113.189:35697: connect: Connection refused (error 111)
W20260809 22:58:19.010615 18107 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260809 22:58:19.010928 18107 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:58:19.011023 18107 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:58:19.011137 18107 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:58:19.018086 18107 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 22:58:19.018168 18107 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:58:19.018257 18107 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:58:19.018321 18107 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 22:58:19.018407 18107 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 22:58:19.027688 18107 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.113.148:43367
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786315935617919-15814-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786315935617919-15814-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.15.113.190:38393,127.15.113.189:35697,127.15.113.188:45579
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.189.keytab
--principal=kudu/127.15.113.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.113.189:35697
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786315935617919-15814-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.15.113.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786315935617919-15814-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:58:19.028843 18107 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:58:19.030129 18107 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:58:19.041303 18115 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:58:19.041280 18117 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:58:19.041448 18119 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:58:19.042024 18107 server_base.cc:1034] running on GCE node
I20260809 22:58:19.042759 18107 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:58:19.044037 18107 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:58:19.045256 18107 hybrid_clock.cc:648] HybridClock initialized: now 1786316299045188 us; error 43 us; skew 500 ppm
I20260809 22:58:19.045550 18107 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:58:19 dist-test-slave-ncp9 krb5kdc[17974](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316299, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.113.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:58:19.048504 18107 init.cc:377] Logged in from keytab as kudu/127.15.113.189@KRBTEST.COM (short username kudu)
I20260809 22:58:19.049814 18107 webserver.cc:460] Webserver started at http://127.15.113.189:42033/ using document root <none> and password file <none>
I20260809 22:58:19.050484 18107 fs_manager.cc:359] Metadata directory not provided
I20260809 22:58:19.050614 18107 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:58:19.050930 18107 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:58:19.053747 18107 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786315935617919-15814-0/minicluster-data/master-1/data/instance:
uuid: "26e75efb43b54b7eb266fb46dc785088"
format_stamp: "Formatted at 2026-08-09 22:58:19 on dist-test-slave-ncp9"
server_key: "d091ec6b57174df3d7dcee85635cc6b2"
server_key_iv: "3894ada7a37b89102d92b19d6b4757bf"
server_key_version: "encryptionkey@0"
I20260809 22:58:19.054482 18107 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786315935617919-15814-0/minicluster-data/master-1/wal/instance:
uuid: "26e75efb43b54b7eb266fb46dc785088"
format_stamp: "Formatted at 2026-08-09 22:58:19 on dist-test-slave-ncp9"
server_key: "d091ec6b57174df3d7dcee85635cc6b2"
server_key_iv: "3894ada7a37b89102d92b19d6b4757bf"
server_key_version: "encryptionkey@0"
I20260809 22:58:19.059247 18107 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.005s
I20260809 22:58:19.062443 18129 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:58:19.063664 18107 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260809 22:58:19.063817 18107 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786315935617919-15814-0/minicluster-data/master-1/data,/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786315935617919-15814-0/minicluster-data/master-1/wal
uuid: "26e75efb43b54b7eb266fb46dc785088"
format_stamp: "Formatted at 2026-08-09 22:58:19 on dist-test-slave-ncp9"
server_key: "d091ec6b57174df3d7dcee85635cc6b2"
server_key_iv: "3894ada7a37b89102d92b19d6b4757bf"
server_key_version: "encryptionkey@0"
I20260809 22:58:19.064006 18107 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786315935617919-15814-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786315935617919-15814-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786315935617919-15814-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:58:19.076710 18107 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:58:19.080677 18107 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:58:19.080974 18107 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:58:19.098034 18107 rpc_server.cc:307] RPC server started. Bound to: 127.15.113.189:35697
I20260809 22:58:19.098053 18212 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.113.189:35697 every 8 connection(s)
I20260809 22:58:19.099323 18107 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786315935617919-15814-0/minicluster-data/master-1/data/info.pb
I20260809 22:58:19.101380 15814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskItYPJR/build/asan/bin/kudu as pid 18107
I20260809 22:58:19.101542 15814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786315935617919-15814-0/minicluster-data/master-1/wal/instance
I20260809 22:58:19.101974 15814 external_mini_cluster.cc:1428] Setting key fabbc6417d3d67d9fdf6c4af4976ec98
I20260809 22:58:19.102955 18213 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:58:19.109127 18213 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.113.190" port: 38393 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 22:58:19 dist-test-slave-ncp9 krb5kdc[17974](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316299, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.113.189@KRBTEST.COM for kudu/127.15.113.190@KRBTEST.COM
I20260809 22:58:19.128849 18099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:58:19.111626 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51198 (local address 127.15.113.190:38393)
0809 22:58:19.111838 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0809 22:58:19.111845 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:58:19.112882 (+ 1037us) server_negotiation.cc:407] Connection header received
0809 22:58:19.114721 (+ 1839us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:58:19.114727 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:58:19.114807 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:58:19.114914 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:58:19.116285 (+ 1371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:58:19.117191 (+ 906us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:58:19.118982 (+ 1791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:58:19.119388 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:58:19.119482 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:58:19.123326 (+ 3844us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:58:19.123332 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:58:19.123335 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:58:19.123368 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:58:19.125915 (+ 2547us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:58:19.126743 (+ 828us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:58:19.126747 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:58:19.126749 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:58:19.126856 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:58:19.127258 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:58:19.127261 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:58:19.127263 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:58:19.127515 (+ 252us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:58:19.127618 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0809 22:58:19.128455 (+ 837us) server_negotiation.cc:299] Negotiation successful
0809 22:58:19.128611 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260809 22:58:19.130103 18215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:58:19.111965 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:38393 (local address 127.0.0.1:51198)
0809 22:58:19.112702 (+ 737us) negotiation.cc:107] Waiting for socket to connect
0809 22:58:19.112756 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0809 22:58:19.114457 (+ 1701us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:58:19.115086 (+ 629us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:58:19.115094 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:58:19.115579 (+ 485us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:58:19.116103 (+ 524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:58:19.116119 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:58:19.117356 (+ 1237us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:58:19.117361 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:58:19.118826 (+ 1465us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:58:19.118843 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:58:19.119609 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:58:19.119615 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:58:19.119877 (+ 262us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:58:19.120544 (+ 667us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:58:19.120570 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:58:19.123095 (+ 2525us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:58:19.126046 (+ 2951us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:58:19.126054 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:58:19.126068 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:58:19.126647 (+ 579us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:58:19.126948 (+ 301us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:58:19.126952 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:58:19.126955 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:58:19.127159 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:58:19.127648 (+ 489us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:58:19.127656 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:58:19.128086 (+ 430us) client_negotiation.cc:769] Sending connection context
0809 22:58:19.128380 (+ 294us) client_negotiation.cc:240] Negotiation successful
0809 22:58:19.128818 (+ 438us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":483,"thread_start_us":301,"threads_started":1}
I20260809 22:58:19.131824 18213 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.113.189" port: 35697 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.15.113.188@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.113.188@KRBTEST.COM" created.
Aug 09 22:58:19 dist-test-slave-ncp9 krb5kdc[17974](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316299, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.113.189@KRBTEST.COM for kudu/127.15.113.189@KRBTEST.COM
I20260809 22:58:19.145889 18215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:58:19.132610 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.189:35697 (local address 127.0.0.1:55082)
0809 22:58:19.132787 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0809 22:58:19.132806 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 22:58:19.132928 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:58:19.133874 (+ 946us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:58:19.133880 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:58:19.134339 (+ 459us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:58:19.134517 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:58:19.134528 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:58:19.135864 (+ 1336us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:58:19.135870 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:58:19.137128 (+ 1258us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:58:19.137141 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:58:19.137964 (+ 823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:58:19.137969 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:58:19.138088 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:58:19.138556 (+ 468us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:58:19.138572 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:58:19.140575 (+ 2003us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:58:19.143513 (+ 2938us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:58:19.143518 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:58:19.143521 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:58:19.143946 (+ 425us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:58:19.144267 (+ 321us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:58:19.144271 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:58:19.144274 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:58:19.144439 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:58:19.145004 (+ 565us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:58:19.145008 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:58:19.145141 (+ 133us) client_negotiation.cc:769] Sending connection context
0809 22:58:19.145290 (+ 149us) client_negotiation.cc:240] Negotiation successful
0809 22:58:19.145539 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":61}
I20260809 22:58:19.145936 18218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:58:19.132839 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55082 (local address 127.15.113.189:35697)
0809 22:58:19.133477 (+ 638us) server_negotiation.cc:206] Beginning negotiation
0809 22:58:19.133483 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:58:19.133506 (+ 23us) server_negotiation.cc:407] Connection header received
0809 22:58:19.133545 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:58:19.133550 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:58:19.133708 (+ 158us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:58:19.133817 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:58:19.134683 (+ 866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:58:19.135718 (+ 1035us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:58:19.137251 (+ 1533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:58:19.137844 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:58:19.137935 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:58:19.140787 (+ 2852us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:58:19.140797 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:58:19.140808 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:58:19.140835 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:58:19.143381 (+ 2546us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:58:19.144056 (+ 675us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:58:19.144062 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:58:19.144064 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:58:19.144170 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:58:19.144537 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:58:19.144540 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:58:19.144542 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:58:19.144860 (+ 318us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:58:19.145063 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0809 22:58:19.145391 (+ 328us) server_negotiation.cc:299] Negotiation successful
0809 22:58:19.145631 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":55,"server-negotiator.queue_time_us":499,"thread_start_us":374,"threads_started":1}
I20260809 22:58:19.148458 18213 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.113.188" port: 45579 } has no permanent_uuid. Determining permanent_uuid...
I20260809 22:58:19.149614 18215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:58:19.149178 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.188:45579 (local address 127.0.0.1:48674)
0809 22:58:19.149333 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0809 22:58:19.149464 (+ 131us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.113.188:45579: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260809 22:58:19.149878 18130 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.15.113.188:45579: connect: Connection refused (error 111)
I20260809 22:58:19.151331 18215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:58:19.151003 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.188:45579 (local address 127.0.0.1:48682)
0809 22:58:19.151144 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0809 22:58:19.151230 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.113.188:45579: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260809 22:58:19.151611 18213 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.113.188:45579: Network error: Client connection negotiation failed: client connection to 127.15.113.188:45579: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.113.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.188.keytab.
Entry for principal kudu/127.15.113.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.188.keytab.
I20260809 22:58:19.176460 15814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskItYPJR/build/asan/bin/kudu
/tmp/dist-test-taskItYPJR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786315935617919-15814-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786315935617919-15814-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786315935617919-15814-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786315935617919-15814-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.113.188:45579
--webserver_interface=127.15.113.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.188.keytab
--principal=kudu/127.15.113.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.113.148:43367
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.113.190:38393,127.15.113.189:35697,127.15.113.188:45579
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786315935617919-15814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786315935617919-15814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786315935617919-15814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 22:58:19.200878 18213 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.113.188" port: 45579 } attempt: 1
I20260809 22:58:19.202029 18215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:58:19.201601 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.188:45579 (local address 127.0.0.1:48692)
0809 22:58:19.201785 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0809 22:58:19.201884 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.113.188:45579: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
I20260809 22:58:19.202950 18215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:58:19.202618 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.188:45579 (local address 127.0.0.1:48706)
0809 22:58:19.202772 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0809 22:58:19.202847 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.113.188:45579: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260809 22:58:19.203137 18213 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.113.188:45579: Network error: Client connection negotiation failed: client connection to 127.15.113.188:45579: connect: Connection refused (error 111)
I20260809 22:58:19.271327 18213 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.113.188" port: 45579 } attempt: 2
I20260809 22:58:19.272408 18215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:58:19.271981 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.188:45579 (local address 127.0.0.1:48712)
0809 22:58:19.272163 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0809 22:58:19.272253 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.113.188:45579: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260809 22:58:19.273342 18215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:58:19.273017 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.188:45579 (local address 127.0.0.1:48722)
0809 22:58:19.273163 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0809 22:58:19.273231 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.113.188:45579: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260809 22:58:19.273537 18213 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.113.188:45579: Network error: Client connection negotiation failed: client connection to 127.15.113.188:45579: connect: Connection refused (error 111)
I20260809 22:58:19.297592 18096 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.113.189" port: 35697 } attempt: 5
Aug 09 22:58:19 dist-test-slave-ncp9 krb5kdc[17974](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316298, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.113.190@KRBTEST.COM for kudu/127.15.113.189@KRBTEST.COM
I20260809 22:58:19.311544 18098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:58:19.298141 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.189:35697 (local address 127.0.0.1:55088)
0809 22:58:19.298363 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0809 22:58:19.298384 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0809 22:58:19.298518 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:58:19.298846 (+ 328us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:58:19.298850 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:58:19.299377 (+ 527us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:58:19.299620 (+ 243us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:58:19.299633 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:58:19.300562 (+ 929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:58:19.300568 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:58:19.301969 (+ 1401us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:58:19.301986 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:58:19.302691 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:58:19.302698 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:58:19.302942 (+ 244us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:58:19.303584 (+ 642us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:58:19.303609 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:58:19.305916 (+ 2307us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:58:19.309471 (+ 3555us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:58:19.309475 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:58:19.309477 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:58:19.309827 (+ 350us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:58:19.310213 (+ 386us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:58:19.310219 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:58:19.310222 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:58:19.310351 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:58:19.310904 (+ 553us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:58:19.310907 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:58:19.311025 (+ 118us) client_negotiation.cc:769] Sending connection context
0809 22:58:19.311128 (+ 103us) client_negotiation.cc:240] Negotiation successful
0809 22:58:19.311319 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":71}
I20260809 22:58:19.311767 18218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:58:19.298370 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55088 (local address 127.15.113.189:35697)
0809 22:58:19.298510 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0809 22:58:19.298514 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 22:58:19.298533 (+ 19us) server_negotiation.cc:407] Connection header received
0809 22:58:19.298632 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:58:19.298635 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:58:19.298706 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:58:19.298785 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:58:19.299747 (+ 962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:58:19.300428 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:58:19.302100 (+ 1672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:58:19.302540 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:58:19.302628 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:58:19.306107 (+ 3479us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:58:19.306114 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:58:19.306118 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:58:19.306158 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:58:19.309339 (+ 3181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:58:19.309936 (+ 597us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:58:19.309942 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:58:19.309945 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:58:19.310092 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:58:19.310445 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:58:19.310450 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:58:19.310453 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:58:19.310800 (+ 347us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:58:19.310953 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0809 22:58:19.311256 (+ 303us) server_negotiation.cc:299] Negotiation successful
0809 22:58:19.311480 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260809 22:58:19.312729 18096 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.113.188" port: 45579 } has no permanent_uuid. Determining permanent_uuid...
I20260809 22:58:19.313800 18098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:58:19.313375 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.188:45579 (local address 127.0.0.1:48724)
0809 22:58:19.313547 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0809 22:58:19.313641 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.113.188:45579: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260809 22:58:19.314785 18098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:58:19.314396 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.188:45579 (local address 127.0.0.1:48730)
0809 22:58:19.314553 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0809 22:58:19.314642 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.113.188:45579: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260809 22:58:19.315026 18096 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.113.188:45579: Network error: Client connection negotiation failed: client connection to 127.15.113.188:45579: connect: Connection refused (error 111)
I20260809 22:58:19.364732 18213 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.113.188" port: 45579 } attempt: 3
I20260809 22:58:19.365788 18215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:58:19.365348 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.188:45579 (local address 127.0.0.1:48740)
0809 22:58:19.365531 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0809 22:58:19.365631 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.113.188:45579: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260809 22:58:19.366230 18096 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.113.188" port: 45579 } attempt: 1
I20260809 22:58:19.367066 18215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:58:19.366534 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.188:45579 (local address 127.0.0.1:48748)
0809 22:58:19.366823 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0809 22:58:19.366922 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.113.188:45579: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":178}
I20260809 22:58:19.367277 18098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:58:19.366718 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.188:45579 (local address 127.0.0.1:48754)
0809 22:58:19.366955 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0809 22:58:19.367064 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.113.188:45579: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98}
W20260809 22:58:19.367314 18213 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.113.188:45579: Network error: Client connection negotiation failed: client connection to 127.15.113.188:45579: connect: Connection refused (error 111)
I20260809 22:58:19.368340 18098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:58:19.367956 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.188:45579 (local address 127.0.0.1:48760)
0809 22:58:19.368113 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0809 22:58:19.368211 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.113.188:45579: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260809 22:58:19.368539 18096 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.113.188:45579: Network error: Client connection negotiation failed: client connection to 127.15.113.188:45579: connect: Connection refused (error 111)
I20260809 22:58:19.436741 18096 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.113.188" port: 45579 } attempt: 2
I20260809 22:58:19.437922 18098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:58:19.437465 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.188:45579 (local address 127.0.0.1:48770)
0809 22:58:19.437675 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0809 22:58:19.437774 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.113.188:45579: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260809 22:58:19.438843 18098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:58:19.438556 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.188:45579 (local address 127.0.0.1:48776)
0809 22:58:19.438678 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0809 22:58:19.438743 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.113.188:45579: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260809 22:58:19.439046 18096 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.113.188:45579: Network error: Client connection negotiation failed: client connection to 127.15.113.188:45579: connect: Connection refused (error 111)
I20260809 22:58:19.510512 18213 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.113.188" port: 45579 } attempt: 4
I20260809 22:58:19.511554 18215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:58:19.511091 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.188:45579 (local address 127.0.0.1:48786)
0809 22:58:19.511311 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0809 22:58:19.511397 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.113.188:45579: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260809 22:58:19.511485 18226 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260809 22:58:19.511853 18226 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:58:19.511962 18226 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:58:19.512086 18226 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260809 22:58:19.512635 18215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:58:19.512245 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.188:45579 (local address 127.0.0.1:48790)
0809 22:58:19.512410 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0809 22:58:19.512503 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.113.188:45579: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260809 22:58:19.512866 18213 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.113.188:45579: Network error: Client connection negotiation failed: client connection to 127.15.113.188:45579: connect: Connection refused (error 111)
W20260809 22:58:19.519488 18226 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 22:58:19.519569 18226 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:58:19.519634 18226 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:58:19.519690 18226 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 22:58:19.519750 18226 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 22:58:19.529440 18226 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.113.148:43367
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786315935617919-15814-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786315935617919-15814-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.15.113.190:38393,127.15.113.189:35697,127.15.113.188:45579
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.188.keytab
--principal=kudu/127.15.113.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.113.188:45579
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786315935617919-15814-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.15.113.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786315935617919-15814-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:58:19.530602 18226 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:58:19.531880 18226 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:58:19.543059 18240 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:58:19.543089 18236 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:58:19.543313 18234 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:58:19.543852 18226 server_base.cc:1034] running on GCE node
I20260809 22:58:19.544557 18226 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:58:19.545228 18096 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.113.188" port: 45579 } attempt: 3
I20260809 22:58:19.545964 18226 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:58:19.546306 18098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:58:19.545761 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.188:45579 (local address 127.0.0.1:48792)
0809 22:58:19.545991 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0809 22:58:19.546093 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.113.188:45579: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81}
I20260809 22:58:19.547225 18226 hybrid_clock.cc:648] HybridClock initialized: now 1786316299547155 us; error 33 us; skew 500 ppm
I20260809 22:58:19.547260 18098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:58:19.546899 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.188:45579 (local address 127.0.0.1:48796)
0809 22:58:19.547054 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0809 22:58:19.547135 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.113.188:45579: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
W20260809 22:58:19.547504 18096 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.113.188:45579: Network error: Client connection negotiation failed: client connection to 127.15.113.188:45579: connect: Connection refused (error 111)
I20260809 22:58:19.547600 18226 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:58:19 dist-test-slave-ncp9 krb5kdc[17974](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316299, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.113.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:58:19.550688 18226 init.cc:377] Logged in from keytab as kudu/127.15.113.188@KRBTEST.COM (short username kudu)
I20260809 22:58:19.551964 18226 webserver.cc:460] Webserver started at http://127.15.113.188:37747/ using document root <none> and password file <none>
I20260809 22:58:19.552600 18226 fs_manager.cc:359] Metadata directory not provided
I20260809 22:58:19.552683 18226 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:58:19.553018 18226 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:58:19.555822 18226 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786315935617919-15814-0/minicluster-data/master-2/data/instance:
uuid: "c9e4bb40e24e4106a4adf6d629e63dcc"
format_stamp: "Formatted at 2026-08-09 22:58:19 on dist-test-slave-ncp9"
server_key: "cbacae913ab5dea28a2a9578b4d01130"
server_key_iv: "d447ba17c569c619a0ead23761806638"
server_key_version: "encryptionkey@0"
I20260809 22:58:19.556591 18226 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786315935617919-15814-0/minicluster-data/master-2/wal/instance:
uuid: "c9e4bb40e24e4106a4adf6d629e63dcc"
format_stamp: "Formatted at 2026-08-09 22:58:19 on dist-test-slave-ncp9"
server_key: "cbacae913ab5dea28a2a9578b4d01130"
server_key_iv: "d447ba17c569c619a0ead23761806638"
server_key_version: "encryptionkey@0"
I20260809 22:58:19.561436 18226 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.004s
I20260809 22:58:19.564915 18250 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:58:19.566166 18226 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 22:58:19.566293 18226 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786315935617919-15814-0/minicluster-data/master-2/data,/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786315935617919-15814-0/minicluster-data/master-2/wal
uuid: "c9e4bb40e24e4106a4adf6d629e63dcc"
format_stamp: "Formatted at 2026-08-09 22:58:19 on dist-test-slave-ncp9"
server_key: "cbacae913ab5dea28a2a9578b4d01130"
server_key_iv: "d447ba17c569c619a0ead23761806638"
server_key_version: "encryptionkey@0"
I20260809 22:58:19.566461 18226 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786315935617919-15814-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786315935617919-15814-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786315935617919-15814-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:58:19.606799 18226 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:58:19.610841 18226 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:58:19.611056 18226 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:58:19.628264 18226 rpc_server.cc:307] RPC server started. Bound to: 127.15.113.188:45579
I20260809 22:58:19.628284 18324 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.113.188:45579 every 8 connection(s)
I20260809 22:58:19.629567 18226 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786315935617919-15814-0/minicluster-data/master-2/data/info.pb
I20260809 22:58:19.633291 18326 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:58:19.634234 15814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskItYPJR/build/asan/bin/kudu as pid 18226
I20260809 22:58:19.634394 15814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786315935617919-15814-0/minicluster-data/master-2/wal/instance
I20260809 22:58:19.634806 15814 external_mini_cluster.cc:1428] Setting key e18684bb109ff488a000bf529efa3b1a
I20260809 22:58:19.642712 18326 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.113.190" port: 38393 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 22:58:19 dist-test-slave-ncp9 krb5kdc[17974](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316298, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.113.188@KRBTEST.COM
Aug 09 22:58:19 dist-test-slave-ncp9 krb5kdc[17974](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316299, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.113.188@KRBTEST.COM for kudu/127.15.113.190@KRBTEST.COM
I20260809 22:58:19.658716 18333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:58:19.636669 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48806 (local address 127.15.113.188:45579)
0809 22:58:19.637474 (+ 805us) server_negotiation.cc:206] Beginning negotiation
0809 22:58:19.637489 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0809 22:58:19.637539 (+ 50us) server_negotiation.cc:407] Connection header received
0809 22:58:19.639593 (+ 2054us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:58:19.639621 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:58:19.640118 (+ 497us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:58:19.640552 (+ 434us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:58:19.641883 (+ 1331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:58:19.643894 (+ 2011us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:58:19.645345 (+ 1451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:58:19.646198 (+ 853us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:58:19.646366 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:58:19.651149 (+ 4783us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:58:19.651159 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:58:19.651182 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:58:19.651245 (+ 63us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:58:19.654307 (+ 3062us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:58:19.655084 (+ 777us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:58:19.655093 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:58:19.655101 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:58:19.655284 (+ 183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:58:19.655599 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:58:19.655603 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:58:19.655605 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:58:19.656271 (+ 666us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:58:19.656580 (+ 309us) server_negotiation.cc:1035] Waiting for connection context
0809 22:58:19.656925 (+ 345us) server_negotiation.cc:299] Negotiation successful
0809 22:58:19.657342 (+ 417us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":561,"thread_start_us":338,"threads_started":1}
I20260809 22:58:19.660813 18339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:58:19.645509 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:38393 (local address 127.0.0.1:51204)
0809 22:58:19.646316 (+ 807us) negotiation.cc:107] Waiting for socket to connect
0809 22:58:19.646358 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0809 22:58:19.646497 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:58:19.646928 (+ 431us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:58:19.646935 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:58:19.647557 (+ 622us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:58:19.647810 (+ 253us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:58:19.647819 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:58:19.649294 (+ 1475us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:58:19.649300 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:58:19.650835 (+ 1535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:58:19.650851 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:58:19.651769 (+ 918us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:58:19.651774 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:58:19.651939 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:58:19.652482 (+ 543us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:58:19.652505 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:58:19.654766 (+ 2261us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:58:19.658449 (+ 3683us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:58:19.658479 (+ 30us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:58:19.658487 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:58:19.659058 (+ 571us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:58:19.659404 (+ 346us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:58:19.659408 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:58:19.659411 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:58:19.659559 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:58:19.660015 (+ 456us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:58:19.660022 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:58:19.660176 (+ 154us) client_negotiation.cc:769] Sending connection context
0809 22:58:19.660320 (+ 144us) client_negotiation.cc:240] Negotiation successful
0809 22:58:19.660537 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":578,"thread_start_us":431,"threads_started":1}
I20260809 22:58:19.661084 18338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:58:19.645447 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51204 (local address 127.15.113.190:38393)
0809 22:58:19.645957 (+ 510us) server_negotiation.cc:206] Beginning negotiation
0809 22:58:19.645963 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:58:19.646589 (+ 626us) server_negotiation.cc:407] Connection header received
0809 22:58:19.646659 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:58:19.646676 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:58:19.646756 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:58:19.646872 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:58:19.647969 (+ 1097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:58:19.649151 (+ 1182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:58:19.650969 (+ 1818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:58:19.651585 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:58:19.651724 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:58:19.654941 (+ 3217us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:58:19.654946 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:58:19.654949 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:58:19.654981 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:58:19.658301 (+ 3320us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:58:19.659162 (+ 861us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:58:19.659166 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:58:19.659170 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:58:19.659291 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:58:19.659652 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:58:19.659655 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:58:19.659657 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:58:19.659899 (+ 242us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:58:19.660009 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0809 22:58:19.660460 (+ 451us) server_negotiation.cc:299] Negotiation successful
0809 22:58:19.660687 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":371,"thread_start_us":235,"threads_started":1}
I20260809 22:58:19.662343 18326 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.113.189" port: 35697 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 22:58:19 dist-test-slave-ncp9 krb5kdc[17974](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316299, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.113.188@KRBTEST.COM for kudu/127.15.113.189@KRBTEST.COM
Aug 09 22:58:19 dist-test-slave-ncp9 krb5kdc[17974](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316298, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.113.190@KRBTEST.COM
I20260809 22:58:19.675231 18339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:58:19.662928 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.189:35697 (local address 127.0.0.1:55094)
0809 22:58:19.663073 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0809 22:58:19.663090 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 22:58:19.663176 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:58:19.663489 (+ 313us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:58:19.663492 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:58:19.663806 (+ 314us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:58:19.663932 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:58:19.663940 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:58:19.664971 (+ 1031us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:58:19.664976 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:58:19.665850 (+ 874us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:58:19.665862 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:58:19.666551 (+ 689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:58:19.666557 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:58:19.666738 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:58:19.667214 (+ 476us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:58:19.667237 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:58:19.669562 (+ 2325us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:58:19.673131 (+ 3569us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:58:19.673134 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:58:19.673137 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:58:19.673480 (+ 343us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:58:19.673866 (+ 386us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:58:19.673869 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:58:19.673871 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:58:19.674014 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:58:19.674497 (+ 483us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:58:19.674500 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:58:19.674626 (+ 126us) client_negotiation.cc:769] Sending connection context
0809 22:58:19.674763 (+ 137us) client_negotiation.cc:240] Negotiation successful
0809 22:58:19.675013 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":45}
I20260809 22:58:19.675361 18218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:58:19.663033 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55094 (local address 127.15.113.189:35697)
0809 22:58:19.663219 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0809 22:58:19.663225 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:58:19.663244 (+ 19us) server_negotiation.cc:407] Connection header received
0809 22:58:19.663306 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:58:19.663310 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:58:19.663381 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:58:19.663489 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:58:19.664089 (+ 600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:58:19.664807 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:58:19.665989 (+ 1182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:58:19.666404 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:58:19.666511 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:58:19.669750 (+ 3239us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:58:19.669757 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:58:19.669762 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:58:19.669803 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:58:19.673012 (+ 3209us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:58:19.673592 (+ 580us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:58:19.673598 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:58:19.673601 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:58:19.673774 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:58:19.674123 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:58:19.674126 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:58:19.674128 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:58:19.674382 (+ 254us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:58:19.674518 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0809 22:58:19.674843 (+ 325us) server_negotiation.cc:299] Negotiation successful
0809 22:58:19.675018 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260809 22:58:19.676347 18326 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.113.188" port: 45579 } has no permanent_uuid. Determining permanent_uuid...
I20260809 22:58:19.676549 18338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:58:19.662086 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51218 (local address 127.15.113.190:38393)
0809 22:58:19.662603 (+ 517us) server_negotiation.cc:206] Beginning negotiation
0809 22:58:19.662607 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 22:58:19.662620 (+ 13us) server_negotiation.cc:407] Connection header received
0809 22:58:19.662656 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:58:19.662659 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:58:19.662739 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:58:19.662828 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:58:19.664224 (+ 1396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:58:19.664941 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:58:19.666106 (+ 1165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:58:19.666498 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:58:19.666598 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:58:19.671045 (+ 4447us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:58:19.671051 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:58:19.671056 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:58:19.671092 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:58:19.673884 (+ 2792us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:58:19.674464 (+ 580us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:58:19.674469 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:58:19.674473 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:58:19.674602 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:58:19.675006 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:58:19.675011 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:58:19.675013 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:58:19.675347 (+ 334us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:58:19.675601 (+ 254us) server_negotiation.cc:1035] Waiting for connection context
0809 22:58:19.676051 (+ 450us) server_negotiation.cc:299] Negotiation successful
0809 22:58:19.676268 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":407}
Aug 09 22:58:19 dist-test-slave-ncp9 krb5kdc[17974](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316299, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.113.188@KRBTEST.COM for kudu/127.15.113.188@KRBTEST.COM
Aug 09 22:58:19 dist-test-slave-ncp9 krb5kdc[17974](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316298, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.113.189@KRBTEST.COM
I20260809 22:58:19.689210 18339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:58:19.677168 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.188:45579 (local address 127.0.0.1:48818)
0809 22:58:19.677317 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0809 22:58:19.677334 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 22:58:19.677404 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:58:19.677680 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:58:19.677684 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:58:19.678076 (+ 392us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:58:19.678254 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:58:19.678266 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:58:19.679311 (+ 1045us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:58:19.679317 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:58:19.680349 (+ 1032us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:58:19.680361 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:58:19.681210 (+ 849us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:58:19.681217 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:58:19.681433 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:58:19.681906 (+ 473us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:58:19.681928 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:58:19.683955 (+ 2027us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:58:19.687120 (+ 3165us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:58:19.687123 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:58:19.687126 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:58:19.687469 (+ 343us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:58:19.687803 (+ 334us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:58:19.687807 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:58:19.687809 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:58:19.687920 (+ 111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:58:19.688478 (+ 558us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:58:19.688481 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:58:19.688600 (+ 119us) client_negotiation.cc:769] Sending connection context
0809 22:58:19.688715 (+ 115us) client_negotiation.cc:240] Negotiation successful
0809 22:58:19.688957 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":48}
I20260809 22:58:19.689476 18333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:58:19.677228 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48818 (local address 127.15.113.188:45579)
0809 22:58:19.677384 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0809 22:58:19.677389 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:58:19.677407 (+ 18us) server_negotiation.cc:407] Connection header received
0809 22:58:19.677479 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:58:19.677484 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:58:19.677548 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:58:19.677640 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:58:19.678398 (+ 758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:58:19.679179 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:58:19.680477 (+ 1298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:58:19.681045 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:58:19.681136 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:58:19.684119 (+ 2983us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:58:19.684123 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:58:19.684126 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:58:19.684160 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:58:19.686986 (+ 2826us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:58:19.687568 (+ 582us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:58:19.687573 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:58:19.687576 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:58:19.687708 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:58:19.688018 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:58:19.688023 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:58:19.688026 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:58:19.688355 (+ 329us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:58:19.688487 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0809 22:58:19.688831 (+ 344us) server_negotiation.cc:299] Negotiation successful
0809 22:58:19.689103 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":25,"server-negotiator.queue_time_us":60}
I20260809 22:58:19.692279 18218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:58:19.678222 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55110 (local address 127.15.113.189:35697)
0809 22:58:19.678362 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0809 22:58:19.678366 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 22:58:19.678381 (+ 15us) server_negotiation.cc:407] Connection header received
0809 22:58:19.678482 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:58:19.678487 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:58:19.678549 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:58:19.678641 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:58:19.679877 (+ 1236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:58:19.680597 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:58:19.682108 (+ 1511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:58:19.682527 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:58:19.682617 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:58:19.686742 (+ 4125us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:58:19.686749 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:58:19.686754 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:58:19.686789 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:58:19.689764 (+ 2975us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:58:19.690498 (+ 734us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:58:19.690504 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:58:19.690506 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:58:19.690655 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:58:19.691001 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:58:19.691006 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:58:19.691010 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:58:19.691271 (+ 261us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:58:19.691515 (+ 244us) server_negotiation.cc:1035] Waiting for connection context
0809 22:58:19.691774 (+ 259us) server_negotiation.cc:299] Negotiation successful
0809 22:58:19.691983 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260809 22:58:19.694384 18326 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c9e4bb40e24e4106a4adf6d629e63dcc: Bootstrap starting.
I20260809 22:58:19.696691 15814 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 22:58:19.697830 18326 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c9e4bb40e24e4106a4adf6d629e63dcc: Neither blocks nor log segments found. Creating new log.
I20260809 22:58:19.698820 18326 log.cc:824] T 00000000000000000000000000000000 P c9e4bb40e24e4106a4adf6d629e63dcc: Log is configured to *not* fsync() on all Append() calls
I20260809 22:58:19.701855 18326 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c9e4bb40e24e4106a4adf6d629e63dcc: No bootstrap required, opened a new log
I20260809 22:58:19.699338 18065 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "cf333d32995a493bb7cef3ede9926ce2"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51218
I20260809 22:58:19.712322 18326 raft_consensus.cc:348] T 00000000000000000000000000000000 P c9e4bb40e24e4106a4adf6d629e63dcc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf333d32995a493bb7cef3ede9926ce2" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 38393 } } peers { permanent_uuid: "26e75efb43b54b7eb266fb46dc785088" member_type: VOTER last_known_addr { host: "127.15.113.189" port: 35697 } } peers { permanent_uuid: "c9e4bb40e24e4106a4adf6d629e63dcc" member_type: VOTER last_known_addr { host: "127.15.113.188" port: 45579 } }
I20260809 22:58:19.712716 18326 raft_consensus.cc:729] T 00000000000000000000000000000000 P c9e4bb40e24e4106a4adf6d629e63dcc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c9e4bb40e24e4106a4adf6d629e63dcc, State: Initialized, Role: FOLLOWER
I20260809 22:58:19.713250 18326 consensus_queue.cc:260] T 00000000000000000000000000000000 P c9e4bb40e24e4106a4adf6d629e63dcc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf333d32995a493bb7cef3ede9926ce2" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 38393 } } peers { permanent_uuid: "26e75efb43b54b7eb266fb46dc785088" member_type: VOTER last_known_addr { host: "127.15.113.189" port: 35697 } } peers { permanent_uuid: "c9e4bb40e24e4106a4adf6d629e63dcc" member_type: VOTER last_known_addr { host: "127.15.113.188" port: 45579 } }
I20260809 22:58:19.714134 18343 sys_catalog.cc:455] T 00000000000000000000000000000000 P c9e4bb40e24e4106a4adf6d629e63dcc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf333d32995a493bb7cef3ede9926ce2" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 38393 } } peers { permanent_uuid: "26e75efb43b54b7eb266fb46dc785088" member_type: VOTER last_known_addr { host: "127.15.113.189" port: 35697 } } peers { permanent_uuid: "c9e4bb40e24e4106a4adf6d629e63dcc" member_type: VOTER last_known_addr { host: "127.15.113.188" port: 45579 } } }
I20260809 22:58:19.714437 18343 sys_catalog.cc:458] T 00000000000000000000000000000000 P c9e4bb40e24e4106a4adf6d629e63dcc [sys.catalog]: This master's current role is: FOLLOWER
I20260809 22:58:19.715173 18326 sys_catalog.cc:564] T 00000000000000000000000000000000 P c9e4bb40e24e4106a4adf6d629e63dcc [sys.catalog]: configured and running, proceeding with master startup.
W20260809 22:58:19.723990 18360 catalog_manager.cc:1558] T 00000000000000000000000000000000 P c9e4bb40e24e4106a4adf6d629e63dcc: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 22:58:19.724118 18360 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 22:58:19.724726 18096 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.113.188" port: 45579 } attempt: 4
Aug 09 22:58:19 dist-test-slave-ncp9 krb5kdc[17974](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316298, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.113.190@KRBTEST.COM for kudu/127.15.113.188@KRBTEST.COM
I20260809 22:58:19.722000 18226 master_runner.cc:186] Error getting master registration for 127.15.113.190:38393: OK, instance_id { permanent_uuid: "cf333d32995a493bb7cef3ede9926ce2" instance_seqno: 1786316298590161 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 22:58:19.733556 18226 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260809 22:58:19.740283 18098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:58:19.725307 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.188:45579 (local address 127.0.0.1:48834)
0809 22:58:19.725470 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0809 22:58:19.725488 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 22:58:19.725577 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:58:19.725912 (+ 335us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:58:19.725918 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:58:19.726429 (+ 511us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:58:19.726638 (+ 209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:58:19.726650 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:58:19.727764 (+ 1114us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:58:19.727771 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:58:19.729262 (+ 1491us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:58:19.729277 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:58:19.730061 (+ 784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:58:19.730067 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:58:19.730260 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:58:19.730942 (+ 682us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:58:19.730966 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:58:19.733746 (+ 2780us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:58:19.737709 (+ 3963us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:58:19.737714 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:58:19.737718 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:58:19.738134 (+ 416us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:58:19.738500 (+ 366us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:58:19.738505 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:58:19.738509 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:58:19.738676 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:58:19.739302 (+ 626us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:58:19.739307 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:58:19.739467 (+ 160us) client_negotiation.cc:769] Sending connection context
0809 22:58:19.739619 (+ 152us) client_negotiation.cc:240] Negotiation successful
0809 22:58:19.739893 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":44}
I20260809 22:58:19.740348 18333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:58:19.725366 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48834 (local address 127.15.113.188:45579)
0809 22:58:19.725555 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0809 22:58:19.725561 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:58:19.725585 (+ 24us) server_negotiation.cc:407] Connection header received
0809 22:58:19.725703 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:58:19.725708 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:58:19.725787 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:58:19.725885 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:58:19.726793 (+ 908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:58:19.727618 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:58:19.729404 (+ 1786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:58:19.729906 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:58:19.730018 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:58:19.733924 (+ 3906us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:58:19.733931 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:58:19.733936 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:58:19.733975 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:58:19.737575 (+ 3600us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:58:19.738254 (+ 679us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:58:19.738259 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:58:19.738264 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:58:19.738403 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:58:19.738789 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:58:19.738794 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:58:19.738797 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:58:19.739178 (+ 381us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:58:19.739347 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0809 22:58:19.739808 (+ 461us) server_negotiation.cc:299] Negotiation successful
0809 22:58:19.740029 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260809 22:58:19.746385 18096 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cf333d32995a493bb7cef3ede9926ce2: Bootstrap starting.
I20260809 22:58:19.750408 18096 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cf333d32995a493bb7cef3ede9926ce2: Neither blocks nor log segments found. Creating new log.
I20260809 22:58:19.751729 18096 log.cc:824] T 00000000000000000000000000000000 P cf333d32995a493bb7cef3ede9926ce2: Log is configured to *not* fsync() on all Append() calls
I20260809 22:58:19.755007 18096 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cf333d32995a493bb7cef3ede9926ce2: No bootstrap required, opened a new log
I20260809 22:58:19.762451 18096 raft_consensus.cc:348] T 00000000000000000000000000000000 P cf333d32995a493bb7cef3ede9926ce2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf333d32995a493bb7cef3ede9926ce2" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 38393 } } peers { permanent_uuid: "26e75efb43b54b7eb266fb46dc785088" member_type: VOTER last_known_addr { host: "127.15.113.189" port: 35697 } } peers { permanent_uuid: "c9e4bb40e24e4106a4adf6d629e63dcc" member_type: VOTER last_known_addr { host: "127.15.113.188" port: 45579 } }
I20260809 22:58:19.762859 18096 raft_consensus.cc:729] T 00000000000000000000000000000000 P cf333d32995a493bb7cef3ede9926ce2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cf333d32995a493bb7cef3ede9926ce2, State: Initialized, Role: FOLLOWER
I20260809 22:58:19.763340 18096 consensus_queue.cc:260] T 00000000000000000000000000000000 P cf333d32995a493bb7cef3ede9926ce2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf333d32995a493bb7cef3ede9926ce2" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 38393 } } peers { permanent_uuid: "26e75efb43b54b7eb266fb46dc785088" member_type: VOTER last_known_addr { host: "127.15.113.189" port: 35697 } } peers { permanent_uuid: "c9e4bb40e24e4106a4adf6d629e63dcc" member_type: VOTER last_known_addr { host: "127.15.113.188" port: 45579 } }
I20260809 22:58:19.764319 18363 sys_catalog.cc:455] T 00000000000000000000000000000000 P cf333d32995a493bb7cef3ede9926ce2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf333d32995a493bb7cef3ede9926ce2" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 38393 } } peers { permanent_uuid: "26e75efb43b54b7eb266fb46dc785088" member_type: VOTER last_known_addr { host: "127.15.113.189" port: 35697 } } peers { permanent_uuid: "c9e4bb40e24e4106a4adf6d629e63dcc" member_type: VOTER last_known_addr { host: "127.15.113.188" port: 45579 } } }
I20260809 22:58:19.764714 18363 sys_catalog.cc:458] T 00000000000000000000000000000000 P cf333d32995a493bb7cef3ede9926ce2 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 22:58:19.765542 18096 sys_catalog.cc:564] T 00000000000000000000000000000000 P cf333d32995a493bb7cef3ede9926ce2 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 22:58:19.765604 18065 raft_consensus.cc:482] T 00000000000000000000000000000000 P cf333d32995a493bb7cef3ede9926ce2 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260809 22:58:19.765800 18065 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cf333d32995a493bb7cef3ede9926ce2 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:58:19.768517 18065 raft_consensus.cc:504] T 00000000000000000000000000000000 P cf333d32995a493bb7cef3ede9926ce2 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf333d32995a493bb7cef3ede9926ce2" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 38393 } } peers { permanent_uuid: "26e75efb43b54b7eb266fb46dc785088" member_type: VOTER last_known_addr { host: "127.15.113.189" port: 35697 } } peers { permanent_uuid: "c9e4bb40e24e4106a4adf6d629e63dcc" member_type: VOTER last_known_addr { host: "127.15.113.188" port: 45579 } }
I20260809 22:58:19.770479 18065 leader_election.cc:288] T 00000000000000000000000000000000 P cf333d32995a493bb7cef3ede9926ce2 [CANDIDATE]: Term 1 election: Requested vote from peers 26e75efb43b54b7eb266fb46dc785088 (127.15.113.189:35697), c9e4bb40e24e4106a4adf6d629e63dcc (127.15.113.188:45579)
I20260809 22:58:19.770923 18289 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "cf333d32995a493bb7cef3ede9926ce2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "c9e4bb40e24e4106a4adf6d629e63dcc"
I20260809 22:58:19.771440 18289 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c9e4bb40e24e4106a4adf6d629e63dcc [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:58:19.775326 18289 raft_consensus.cc:2454] T 00000000000000000000000000000000 P c9e4bb40e24e4106a4adf6d629e63dcc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf333d32995a493bb7cef3ede9926ce2 in term 1.
I20260809 22:58:19.776062 18024 leader_election.cc:302] T 00000000000000000000000000000000 P cf333d32995a493bb7cef3ede9926ce2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c9e4bb40e24e4106a4adf6d629e63dcc, cf333d32995a493bb7cef3ede9926ce2; no voters:
I20260809 22:58:19.776537 18363 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cf333d32995a493bb7cef3ede9926ce2 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 22:58:19.783131 18363 raft_consensus.cc:686] T 00000000000000000000000000000000 P cf333d32995a493bb7cef3ede9926ce2 [term 1 LEADER]: Becoming Leader. State: Replica: cf333d32995a493bb7cef3ede9926ce2, State: Running, Role: LEADER
I20260809 22:58:19.783807 18363 consensus_queue.cc:237] T 00000000000000000000000000000000 P cf333d32995a493bb7cef3ede9926ce2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf333d32995a493bb7cef3ede9926ce2" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 38393 } } peers { permanent_uuid: "26e75efb43b54b7eb266fb46dc785088" member_type: VOTER last_known_addr { host: "127.15.113.189" port: 35697 } } peers { permanent_uuid: "c9e4bb40e24e4106a4adf6d629e63dcc" member_type: VOTER last_known_addr { host: "127.15.113.188" port: 45579 } }
I20260809 22:58:19.795419 18374 sys_catalog.cc:455] T 00000000000000000000000000000000 P cf333d32995a493bb7cef3ede9926ce2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cf333d32995a493bb7cef3ede9926ce2. Latest consensus state: current_term: 1 leader_uuid: "cf333d32995a493bb7cef3ede9926ce2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf333d32995a493bb7cef3ede9926ce2" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 38393 } } peers { permanent_uuid: "26e75efb43b54b7eb266fb46dc785088" member_type: VOTER last_known_addr { host: "127.15.113.189" port: 35697 } } peers { permanent_uuid: "c9e4bb40e24e4106a4adf6d629e63dcc" member_type: VOTER last_known_addr { host: "127.15.113.188" port: 45579 } } }
I20260809 22:58:19.795850 18374 sys_catalog.cc:458] T 00000000000000000000000000000000 P cf333d32995a493bb7cef3ede9926ce2 [sys.catalog]: This master's current role is: LEADER
I20260809 22:58:19.812084 18213 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.113.188" port: 45579 } attempt: 5
I20260809 22:58:19.812347 18390 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
W20260809 22:58:19.830852 18032 tablet.cc:2369] T 00000000000000000000000000000000 P cf333d32995a493bb7cef3ede9926ce2: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 22:58:19.832170 18338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:58:19.796438 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51234 (local address 127.15.113.190:38393)
0809 22:58:19.796666 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0809 22:58:19.796673 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:58:19.803561 (+ 6888us) server_negotiation.cc:407] Connection header received
0809 22:58:19.804070 (+ 509us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:58:19.804076 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:58:19.804160 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:58:19.804241 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:58:19.806588 (+ 2347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:58:19.807801 (+ 1213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:58:19.809847 (+ 2046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:58:19.810567 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:58:19.810681 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:58:19.824020 (+ 13339us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:58:19.824027 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:58:19.824032 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:58:19.824071 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:58:19.827530 (+ 3459us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:58:19.829027 (+ 1497us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:58:19.829032 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:58:19.829035 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:58:19.829217 (+ 182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:58:19.829956 (+ 739us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:58:19.829961 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:58:19.829964 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:58:19.830319 (+ 355us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:58:19.830587 (+ 268us) server_negotiation.cc:1035] Waiting for connection context
0809 22:58:19.831563 (+ 976us) server_negotiation.cc:299] Negotiation successful
0809 22:58:19.831793 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260809 22:58:19.832175 18218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:58:19.796458 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55114 (local address 127.15.113.189:35697)
0809 22:58:19.796825 (+ 367us) server_negotiation.cc:206] Beginning negotiation
0809 22:58:19.796830 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:58:19.814112 (+ 17282us) server_negotiation.cc:407] Connection header received
0809 22:58:19.814593 (+ 481us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:58:19.814599 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:58:19.814680 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:58:19.814750 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:58:19.817142 (+ 2392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:58:19.818291 (+ 1149us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:58:19.821411 (+ 3120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:58:19.822059 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:58:19.822157 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:58:19.825553 (+ 3396us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:58:19.825558 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:58:19.825563 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:58:19.825601 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:58:19.829031 (+ 3430us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:58:19.829796 (+ 765us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:58:19.829803 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:58:19.829807 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:58:19.829968 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:58:19.830438 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:58:19.830443 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:58:19.830446 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:58:19.830817 (+ 371us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:58:19.831031 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0809 22:58:19.831559 (+ 528us) server_negotiation.cc:299] Negotiation successful
0809 22:58:19.831793 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":244}
I20260809 22:58:19.839949 18390 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 09 22:58:19 dist-test-slave-ncp9 krb5kdc[17974](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316299, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.113.189@KRBTEST.COM for kudu/127.15.113.188@KRBTEST.COM
I20260809 22:58:19.847560 18333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:58:19.798957 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48836 (local address 127.15.113.188:45579)
0809 22:58:19.799138 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0809 22:58:19.799144 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:58:19.820721 (+ 21577us) server_negotiation.cc:407] Connection header received
0809 22:58:19.820912 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:58:19.820919 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:58:19.821001 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:58:19.821122 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:58:19.824533 (+ 3411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:58:19.825702 (+ 1169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:58:19.829590 (+ 3888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:58:19.830327 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:58:19.830428 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:58:19.838139 (+ 7711us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:58:19.838146 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:58:19.838150 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:58:19.838189 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:58:19.841792 (+ 3603us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:58:19.842561 (+ 769us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:58:19.842567 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:58:19.842605 (+ 38us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:58:19.842894 (+ 289us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:58:19.843717 (+ 823us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:58:19.843748 (+ 31us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:58:19.843753 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:58:19.844428 (+ 675us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:58:19.845700 (+ 1272us) server_negotiation.cc:1035] Waiting for connection context
0809 22:58:19.846348 (+ 648us) server_negotiation.cc:299] Negotiation successful
0809 22:58:19.847214 (+ 866us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
W20260809 22:58:19.862948 18260 tablet.cc:2369] T 00000000000000000000000000000000 P c9e4bb40e24e4106a4adf6d629e63dcc: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 22:58:19.868510 18289 raft_consensus.cc:1261] T 00000000000000000000000000000000 P c9e4bb40e24e4106a4adf6d629e63dcc [term 1 FOLLOWER]: Refusing update from remote peer cf333d32995a493bb7cef3ede9926ce2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 22:58:19.869517 18363 consensus_queue.cc:1035] T 00000000000000000000000000000000 P cf333d32995a493bb7cef3ede9926ce2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c9e4bb40e24e4106a4adf6d629e63dcc" member_type: VOTER last_known_addr { host: "127.15.113.188" port: 45579 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 22:58:19.870167 18215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:58:19.812719 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.188:45579 (local address 127.0.0.1:48846)
0809 22:58:19.812999 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0809 22:58:19.813018 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 22:58:19.813121 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:58:19.837564 (+ 24443us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:58:19.837569 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:58:19.838032 (+ 463us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:58:19.838228 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:58:19.838240 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:58:19.839803 (+ 1563us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:58:19.839810 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:58:19.841102 (+ 1292us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:58:19.841117 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:58:19.842838 (+ 1721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:58:19.842845 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:58:19.843045 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:58:19.843603 (+ 558us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:58:19.843626 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:58:19.846231 (+ 2605us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:58:19.864478 (+ 18247us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:58:19.864484 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:58:19.864488 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:58:19.865038 (+ 550us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:58:19.868205 (+ 3167us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:58:19.868210 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:58:19.868214 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:58:19.868389 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:58:19.869171 (+ 782us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:58:19.869177 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:58:19.869340 (+ 163us) client_negotiation.cc:769] Sending connection context
0809 22:58:19.869476 (+ 136us) client_negotiation.cc:240] Negotiation successful
0809 22:58:19.869787 (+ 311us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150}
I20260809 22:58:19.870944 18399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:58:19.813453 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48846 (local address 127.15.113.188:45579)
0809 22:58:19.837294 (+ 23841us) server_negotiation.cc:206] Beginning negotiation
0809 22:58:19.837298 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 22:58:19.837317 (+ 19us) server_negotiation.cc:407] Connection header received
0809 22:58:19.837355 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:58:19.837358 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:58:19.837410 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:58:19.837505 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:58:19.838407 (+ 902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:58:19.839618 (+ 1211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:58:19.841294 (+ 1676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:58:19.842630 (+ 1336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:58:19.846394 (+ 3764us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:58:19.846611 (+ 217us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:58:19.846617 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:58:19.846622 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:58:19.846664 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:58:19.864335 (+ 17671us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:58:19.867882 (+ 3547us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:58:19.867888 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:58:19.867891 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:58:19.868049 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:58:19.868670 (+ 621us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:58:19.868676 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:58:19.868679 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:58:19.869046 (+ 367us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:58:19.869211 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0809 22:58:19.869965 (+ 754us) server_negotiation.cc:299] Negotiation successful
0809 22:58:19.870181 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23736,"thread_start_us":24206,"threads_started":1}
I20260809 22:58:19.853855 17991 master_runner.cc:186] Error getting master registration for 127.15.113.189:35697: OK, instance_id { permanent_uuid: "26e75efb43b54b7eb266fb46dc785088" instance_seqno: 1786316299093200 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 22:58:19.877089 18213 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 26e75efb43b54b7eb266fb46dc785088: Bootstrap starting.
I20260809 22:58:19.882464 18213 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 26e75efb43b54b7eb266fb46dc785088: Neither blocks nor log segments found. Creating new log.
I20260809 22:58:19.883767 18213 log.cc:824] T 00000000000000000000000000000000 P 26e75efb43b54b7eb266fb46dc785088: Log is configured to *not* fsync() on all Append() calls
I20260809 22:58:19.884373 17991 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260809 22:58:19.885908 18343 sys_catalog.cc:455] T 00000000000000000000000000000000 P c9e4bb40e24e4106a4adf6d629e63dcc [sys.catalog]: SysCatalogTable state changed. Reason: New leader cf333d32995a493bb7cef3ede9926ce2. Latest consensus state: current_term: 1 leader_uuid: "cf333d32995a493bb7cef3ede9926ce2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf333d32995a493bb7cef3ede9926ce2" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 38393 } } peers { permanent_uuid: "26e75efb43b54b7eb266fb46dc785088" member_type: VOTER last_known_addr { host: "127.15.113.189" port: 35697 } } peers { permanent_uuid: "c9e4bb40e24e4106a4adf6d629e63dcc" member_type: VOTER last_known_addr { host: "127.15.113.188" port: 45579 } } }
I20260809 22:58:19.886211 18343 sys_catalog.cc:458] T 00000000000000000000000000000000 P c9e4bb40e24e4106a4adf6d629e63dcc [sys.catalog]: This master's current role is: FOLLOWER
I20260809 22:58:19.886989 18213 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 26e75efb43b54b7eb266fb46dc785088: No bootstrap required, opened a new log
I20260809 22:58:19.892072 18402 mvcc.cc:205] Tried to move back new op lower bound from 7316751564249051136 to 7316751563920711680. Current Snapshot: MvccSnapshot[applied={T|T < 7316751564249051136}]
I20260809 22:58:19.894135 18343 sys_catalog.cc:455] T 00000000000000000000000000000000 P c9e4bb40e24e4106a4adf6d629e63dcc [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "cf333d32995a493bb7cef3ede9926ce2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf333d32995a493bb7cef3ede9926ce2" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 38393 } } peers { permanent_uuid: "26e75efb43b54b7eb266fb46dc785088" member_type: VOTER last_known_addr { host: "127.15.113.189" port: 35697 } } peers { permanent_uuid: "c9e4bb40e24e4106a4adf6d629e63dcc" member_type: VOTER last_known_addr { host: "127.15.113.188" port: 45579 } } }
I20260809 22:58:19.894393 18343 sys_catalog.cc:458] T 00000000000000000000000000000000 P c9e4bb40e24e4106a4adf6d629e63dcc [sys.catalog]: This master's current role is: FOLLOWER
I20260809 22:58:19.909420 18374 sys_catalog.cc:455] T 00000000000000000000000000000000 P cf333d32995a493bb7cef3ede9926ce2 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "cf333d32995a493bb7cef3ede9926ce2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf333d32995a493bb7cef3ede9926ce2" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 38393 } } peers { permanent_uuid: "26e75efb43b54b7eb266fb46dc785088" member_type: VOTER last_known_addr { host: "127.15.113.189" port: 35697 } } peers { permanent_uuid: "c9e4bb40e24e4106a4adf6d629e63dcc" member_type: VOTER last_known_addr { host: "127.15.113.188" port: 45579 } } }
I20260809 22:58:19.909701 18374 sys_catalog.cc:458] T 00000000000000000000000000000000 P cf333d32995a493bb7cef3ede9926ce2 [sys.catalog]: This master's current role is: LEADER
I20260809 22:58:19.911005 18390 catalog_manager.cc:1347] Generated new cluster ID: 1e07797994ac4ca097ce1060f2ed3e9e
I20260809 22:58:19.911106 18390 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 22:58:19.928026 18213 raft_consensus.cc:348] T 00000000000000000000000000000000 P 26e75efb43b54b7eb266fb46dc785088 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf333d32995a493bb7cef3ede9926ce2" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 38393 } } peers { permanent_uuid: "26e75efb43b54b7eb266fb46dc785088" member_type: VOTER last_known_addr { host: "127.15.113.189" port: 35697 } } peers { permanent_uuid: "c9e4bb40e24e4106a4adf6d629e63dcc" member_type: VOTER last_known_addr { host: "127.15.113.188" port: 45579 } }
I20260809 22:58:19.928454 18213 raft_consensus.cc:729] T 00000000000000000000000000000000 P 26e75efb43b54b7eb266fb46dc785088 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 26e75efb43b54b7eb266fb46dc785088, State: Initialized, Role: FOLLOWER
I20260809 22:58:19.929013 18213 consensus_queue.cc:260] T 00000000000000000000000000000000 P 26e75efb43b54b7eb266fb46dc785088 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf333d32995a493bb7cef3ede9926ce2" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 38393 } } peers { permanent_uuid: "26e75efb43b54b7eb266fb46dc785088" member_type: VOTER last_known_addr { host: "127.15.113.189" port: 35697 } } peers { permanent_uuid: "c9e4bb40e24e4106a4adf6d629e63dcc" member_type: VOTER last_known_addr { host: "127.15.113.188" port: 45579 } }
I20260809 22:58:19.940392 18412 sys_catalog.cc:455] T 00000000000000000000000000000000 P 26e75efb43b54b7eb266fb46dc785088 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf333d32995a493bb7cef3ede9926ce2" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 38393 } } peers { permanent_uuid: "26e75efb43b54b7eb266fb46dc785088" member_type: VOTER last_known_addr { host: "127.15.113.189" port: 35697 } } peers { permanent_uuid: "c9e4bb40e24e4106a4adf6d629e63dcc" member_type: VOTER last_known_addr { host: "127.15.113.188" port: 45579 } } }
I20260809 22:58:19.940769 18412 sys_catalog.cc:458] T 00000000000000000000000000000000 P 26e75efb43b54b7eb266fb46dc785088 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 22:58:19.942591 18213 sys_catalog.cc:564] T 00000000000000000000000000000000 P 26e75efb43b54b7eb266fb46dc785088 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 22:58:19.942801 18170 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "cf333d32995a493bb7cef3ede9926ce2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "26e75efb43b54b7eb266fb46dc785088"
I20260809 22:58:19.943521 18169 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 26e75efb43b54b7eb266fb46dc785088 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:58:19.947397 18169 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 26e75efb43b54b7eb266fb46dc785088 [term 1 FOLLOWER]: Refusing update from remote peer cf333d32995a493bb7cef3ede9926ce2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 22:58:19.954043 18170 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 26e75efb43b54b7eb266fb46dc785088 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf333d32995a493bb7cef3ede9926ce2 in term 1.
I20260809 22:58:19.956336 18363 consensus_queue.cc:1035] T 00000000000000000000000000000000 P cf333d32995a493bb7cef3ede9926ce2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "26e75efb43b54b7eb266fb46dc785088" member_type: VOTER last_known_addr { host: "127.15.113.189" port: 35697 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 22:58:19.967882 18390 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 22:58:19.971539 18390 catalog_manager.cc:1504] Loading token signing keys...
I20260809 22:58:19.993470 18412 sys_catalog.cc:455] T 00000000000000000000000000000000 P 26e75efb43b54b7eb266fb46dc785088 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cf333d32995a493bb7cef3ede9926ce2. Latest consensus state: current_term: 1 leader_uuid: "cf333d32995a493bb7cef3ede9926ce2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf333d32995a493bb7cef3ede9926ce2" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 38393 } } peers { permanent_uuid: "26e75efb43b54b7eb266fb46dc785088" member_type: VOTER last_known_addr { host: "127.15.113.189" port: 35697 } } peers { permanent_uuid: "c9e4bb40e24e4106a4adf6d629e63dcc" member_type: VOTER last_known_addr { host: "127.15.113.188" port: 45579 } } }
I20260809 22:58:19.993801 18412 sys_catalog.cc:458] T 00000000000000000000000000000000 P 26e75efb43b54b7eb266fb46dc785088 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 22:58:19.995797 18390 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cf333d32995a493bb7cef3ede9926ce2: Generated new TSK 0
I20260809 22:58:19.996840 18390 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 22:58:19.998083 18421 sys_catalog.cc:455] T 00000000000000000000000000000000 P 26e75efb43b54b7eb266fb46dc785088 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "cf333d32995a493bb7cef3ede9926ce2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf333d32995a493bb7cef3ede9926ce2" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 38393 } } peers { permanent_uuid: "26e75efb43b54b7eb266fb46dc785088" member_type: VOTER last_known_addr { host: "127.15.113.189" port: 35697 } } peers { permanent_uuid: "c9e4bb40e24e4106a4adf6d629e63dcc" member_type: VOTER last_known_addr { host: "127.15.113.188" port: 45579 } } }
I20260809 22:58:19.999467 18374 sys_catalog.cc:455] T 00000000000000000000000000000000 P cf333d32995a493bb7cef3ede9926ce2 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "cf333d32995a493bb7cef3ede9926ce2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf333d32995a493bb7cef3ede9926ce2" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 38393 } } peers { permanent_uuid: "26e75efb43b54b7eb266fb46dc785088" member_type: VOTER last_known_addr { host: "127.15.113.189" port: 35697 } } peers { permanent_uuid: "c9e4bb40e24e4106a4adf6d629e63dcc" member_type: VOTER last_known_addr { host: "127.15.113.188" port: 45579 } } }
I20260809 22:58:19.999753 18374 sys_catalog.cc:458] T 00000000000000000000000000000000 P cf333d32995a493bb7cef3ede9926ce2 [sys.catalog]: This master's current role is: LEADER
I20260809 22:58:20.004495 18421 sys_catalog.cc:458] T 00000000000000000000000000000000 P 26e75efb43b54b7eb266fb46dc785088 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 22:58:20.027269 18433 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
W20260809 22:58:20.143612 18338 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:58:20.129740 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51250 (local address 127.15.113.190:38393)
0809 22:58:20.130303 (+ 563us) server_negotiation.cc:206] Beginning negotiation
0809 22:58:20.130309 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:58:20.141486 (+ 11177us) server_negotiation.cc:407] Connection header received
0809 22:58:20.141821 (+ 335us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:58:20.141828 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:58:20.141911 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:58:20.141985 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:58:20.143362 (+ 1377us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51250: BlockingRecv error: recv got EOF from 127.0.0.1:51250 (error 108)
Metrics: {"server-negotiator.queue_time_us":445}
W20260809 22:58:20.143797 18218 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:58:20.129411 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55126 (local address 127.15.113.189:35697)
0809 22:58:20.129969 (+ 558us) server_negotiation.cc:206] Beginning negotiation
0809 22:58:20.129976 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:58:20.141779 (+ 11803us) server_negotiation.cc:407] Connection header received
0809 22:58:20.142011 (+ 232us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:58:20.142029 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:58:20.142106 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:58:20.142174 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:58:20.143582 (+ 1408us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55126: BlockingRecv error: recv got EOF from 127.0.0.1:55126 (error 108)
Metrics: {"server-negotiator.queue_time_us":404}
W20260809 22:58:20.145700 18399 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:58:20.130114 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48862 (local address 127.15.113.188:45579)
0809 22:58:20.130438 (+ 324us) server_negotiation.cc:206] Beginning negotiation
0809 22:58:20.130443 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:58:20.144221 (+ 13778us) server_negotiation.cc:407] Connection header received
0809 22:58:20.144611 (+ 390us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:58:20.144615 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:58:20.144694 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:58:20.144779 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:58:20.145473 (+ 694us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48862: BlockingRecv error: recv got EOF from 127.0.0.1:48862 (error 108)
Metrics: {"server-negotiator.queue_time_us":217}
W20260809 22:58:20.151695 18218 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:58:20.149871 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55138 (local address 127.15.113.189:35697)
0809 22:58:20.150036 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0809 22:58:20.150042 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:58:20.150456 (+ 414us) server_negotiation.cc:407] Connection header received
0809 22:58:20.150655 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:58:20.150659 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:58:20.150740 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:58:20.150831 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:58:20.151447 (+ 616us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55138: BlockingRecv error: recv got EOF from 127.0.0.1:55138 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
W20260809 22:58:20.157516 18218 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:58:20.155666 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55144 (local address 127.15.113.189:35697)
0809 22:58:20.155809 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0809 22:58:20.155814 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:58:20.156319 (+ 505us) server_negotiation.cc:407] Connection header received
0809 22:58:20.156489 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:58:20.156493 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:58:20.156549 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:58:20.156622 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:58:20.157350 (+ 728us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55144: BlockingRecv error: recv got EOF from 127.0.0.1:55144 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
W20260809 22:58:20.164050 18218 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:58:20.162293 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55146 (local address 127.15.113.189:35697)
0809 22:58:20.162447 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0809 22:58:20.162453 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:58:20.162906 (+ 453us) server_negotiation.cc:407] Connection header received
0809 22:58:20.163066 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:58:20.163071 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:58:20.163140 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:58:20.163215 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:58:20.163831 (+ 616us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55146: BlockingRecv error: recv got EOF from 127.0.0.1:55146 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
W20260809 22:58:20.183950 18218 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:58:20.182013 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55158 (local address 127.15.113.189:35697)
0809 22:58:20.182255 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0809 22:58:20.182262 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:58:20.182771 (+ 509us) server_negotiation.cc:407] Connection header received
0809 22:58:20.182937 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:58:20.182943 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:58:20.183010 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:58:20.183089 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:58:20.183738 (+ 649us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55158: BlockingRecv error: recv got EOF from 127.0.0.1:55158 (error 108)
Metrics: {"server-negotiator.queue_time_us":107}
W20260809 22:58:20.201532 18218 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:58:20.199503 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55168 (local address 127.15.113.189:35697)
0809 22:58:20.199679 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0809 22:58:20.199685 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:58:20.200126 (+ 441us) server_negotiation.cc:407] Connection header received
0809 22:58:20.200398 (+ 272us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:58:20.200403 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:58:20.200494 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:58:20.200573 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:58:20.201327 (+ 754us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55168: BlockingRecv error: recv got EOF from 127.0.0.1:55168 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
W20260809 22:58:20.222443 18218 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:58:20.220629 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55180 (local address 127.15.113.189:35697)
0809 22:58:20.220805 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0809 22:58:20.220811 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:58:20.221214 (+ 403us) server_negotiation.cc:407] Connection header received
0809 22:58:20.221386 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:58:20.221390 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:58:20.221463 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:58:20.221554 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:58:20.222238 (+ 684us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55180: BlockingRecv error: recv got EOF from 127.0.0.1:55180 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
W20260809 22:58:20.259855 18218 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:58:20.257747 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55190 (local address 127.15.113.189:35697)
0809 22:58:20.257942 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0809 22:58:20.257949 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:58:20.258459 (+ 510us) server_negotiation.cc:407] Connection header received
0809 22:58:20.258673 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:58:20.258677 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:58:20.258758 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:58:20.258844 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:58:20.259576 (+ 732us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55190: BlockingRecv error: recv got EOF from 127.0.0.1:55190 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
W20260809 22:58:20.328698 18218 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:58:20.326997 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55202 (local address 127.15.113.189:35697)
0809 22:58:20.327203 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0809 22:58:20.327210 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:58:20.327475 (+ 265us) server_negotiation.cc:407] Connection header received
0809 22:58:20.327636 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:58:20.327642 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:58:20.327732 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:58:20.327841 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:58:20.328485 (+ 644us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55202: BlockingRecv error: recv got EOF from 127.0.0.1:55202 (error 108)
Metrics: {"server-negotiator.queue_time_us":76}
W20260809 22:58:20.461500 18218 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:58:20.459792 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55210 (local address 127.15.113.189:35697)
0809 22:58:20.459959 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0809 22:58:20.459964 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:58:20.460318 (+ 354us) server_negotiation.cc:407] Connection header received
0809 22:58:20.460469 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:58:20.460473 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:58:20.460546 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:58:20.460633 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:58:20.461310 (+ 677us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55210: BlockingRecv error: recv got EOF from 127.0.0.1:55210 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
W20260809 22:58:20.724154 18218 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:58:20.722558 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55212 (local address 127.15.113.189:35697)
0809 22:58:20.722744 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0809 22:58:20.722749 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:58:20.722959 (+ 210us) server_negotiation.cc:407] Connection header received
0809 22:58:20.723139 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:58:20.723143 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:58:20.723218 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:58:20.723312 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:58:20.723971 (+ 659us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55212: BlockingRecv error: recv got EOF from 127.0.0.1:55212 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
I20260809 22:58:20.726223 18360 catalog_manager.cc:1259] Loaded cluster ID: 1e07797994ac4ca097ce1060f2ed3e9e
I20260809 22:58:20.726311 18360 catalog_manager.cc:1552] T 00000000000000000000000000000000 P c9e4bb40e24e4106a4adf6d629e63dcc: loading cluster ID for follower catalog manager: success
I20260809 22:58:20.728802 18360 catalog_manager.cc:1574] T 00000000000000000000000000000000 P c9e4bb40e24e4106a4adf6d629e63dcc: acquiring CA information for follower catalog manager: success
I20260809 22:58:20.730545 18360 catalog_manager.cc:1602] T 00000000000000000000000000000000 P c9e4bb40e24e4106a4adf6d629e63dcc: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 22:58:21.031358 18433 catalog_manager.cc:1259] Loaded cluster ID: 1e07797994ac4ca097ce1060f2ed3e9e
I20260809 22:58:21.031481 18433 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 26e75efb43b54b7eb266fb46dc785088: loading cluster ID for follower catalog manager: success
I20260809 22:58:21.034587 18433 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 26e75efb43b54b7eb266fb46dc785088: acquiring CA information for follower catalog manager: success
I20260809 22:58:21.036742 18433 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 26e75efb43b54b7eb266fb46dc785088: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 22:58:21.248944 18542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:58:21.241753 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55220 (local address 127.15.113.189:35697)
0809 22:58:21.242394 (+ 641us) server_negotiation.cc:206] Beginning negotiation
0809 22:58:21.242402 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 22:58:21.242673 (+ 271us) server_negotiation.cc:407] Connection header received
0809 22:58:21.242832 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:58:21.242838 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:58:21.243027 (+ 189us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:58:21.243174 (+ 147us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 22:58:21.243611 (+ 437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:58:21.244812 (+ 1201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:58:21.246155 (+ 1343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:58:21.246843 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:58:21.246983 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:58:21.247449 (+ 466us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 22:58:21.248171 (+ 722us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 22:58:21.248336 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0809 22:58:21.248412 (+ 76us) server_negotiation.cc:299] Negotiation successful
0809 22:58:21.248500 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":436,"thread_start_us":248,"threads_started":1}
I20260809 22:58:21.261981 18551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:58:21.255779 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48874 (local address 127.15.113.188:45579)
0809 22:58:21.256409 (+ 630us) server_negotiation.cc:206] Beginning negotiation
0809 22:58:21.256413 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 22:58:21.256429 (+ 16us) server_negotiation.cc:407] Connection header received
0809 22:58:21.256525 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:58:21.256529 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:58:21.256690 (+ 161us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:58:21.256818 (+ 128us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 22:58:21.257302 (+ 484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:58:21.258052 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:58:21.259906 (+ 1854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:58:21.260288 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:58:21.260366 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:58:21.260961 (+ 595us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 22:58:21.261412 (+ 451us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 22:58:21.261511 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0809 22:58:21.261657 (+ 146us) server_negotiation.cc:299] Negotiation successful
0809 22:58:21.261718 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":534,"thread_start_us":380,"threads_started":1}
I20260809 22:58:21.285506 18542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:58:21.268942 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55234 (local address 127.15.113.189:35697)
0809 22:58:21.269123 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0809 22:58:21.269130 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:58:21.279180 (+ 10050us) server_negotiation.cc:407] Connection header received
0809 22:58:21.279339 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:58:21.279345 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:58:21.279435 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:58:21.279529 (+ 94us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 22:58:21.280060 (+ 531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:58:21.281318 (+ 1258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:58:21.283316 (+ 1998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:58:21.283957 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:58:21.284047 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:58:21.284625 (+ 578us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 22:58:21.284755 (+ 130us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 22:58:21.284821 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0809 22:58:21.285098 (+ 277us) server_negotiation.cc:299] Negotiation successful
0809 22:58:21.285190 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260809 22:58:21.286235 18551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:58:21.268666 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48878 (local address 127.15.113.188:45579)
0809 22:58:21.268827 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0809 22:58:21.268833 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:58:21.278937 (+ 10104us) server_negotiation.cc:407] Connection header received
0809 22:58:21.279837 (+ 900us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:58:21.279842 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:58:21.279924 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:58:21.280021 (+ 97us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 22:58:21.280704 (+ 683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:58:21.281793 (+ 1089us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:58:21.284008 (+ 2215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:58:21.284594 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:58:21.284708 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:58:21.285445 (+ 737us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 22:58:21.285579 (+ 134us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 22:58:21.285660 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0809 22:58:21.285896 (+ 236us) server_negotiation.cc:299] Negotiation successful
0809 22:58:21.285976 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260809 22:58:21.289847 18561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:58:21.268625 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51264 (local address 127.15.113.190:38393)
0809 22:58:21.281903 (+ 13278us) server_negotiation.cc:206] Beginning negotiation
0809 22:58:21.281910 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:58:21.281930 (+ 20us) server_negotiation.cc:407] Connection header received
0809 22:58:21.282047 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:58:21.282052 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:58:21.282259 (+ 207us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:58:21.283251 (+ 992us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 22:58:21.283438 (+ 187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:58:21.284452 (+ 1014us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:58:21.286616 (+ 2164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:58:21.287209 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:58:21.287753 (+ 544us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:58:21.287937 (+ 184us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 22:58:21.289267 (+ 1330us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 22:58:21.289362 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0809 22:58:21.289505 (+ 143us) server_negotiation.cc:299] Negotiation successful
0809 22:58:21.289586 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13158,"thread_start_us":13767,"threads_started":1}
I20260809 22:58:21.295130 15814 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskItYPJR/build/asan/bin/kudu pid 17991
I20260809 22:58:21.296159 18091 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:58:21.405745 15814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskItYPJR/build/asan/bin/kudu with pid 17991
I20260809 22:58:21.434520 15814 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskItYPJR/build/asan/bin/kudu pid 18107
I20260809 22:58:21.435652 18207 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:58:21.543488 15814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskItYPJR/build/asan/bin/kudu with pid 18107
I20260809 22:58:21.568975 15814 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskItYPJR/build/asan/bin/kudu pid 18226
I20260809 22:58:21.570148 18320 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:58:21.675872 15814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskItYPJR/build/asan/bin/kudu with pid 18226
2026-08-09T22:58:21Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (5052 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786315935617919-15814-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 22:58:21 dist-test-slave-ncp9 krb5kdc[18574](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 22:58:21 dist-test-slave-ncp9 krb5kdc[18574](info): set up 2 sockets
Aug 09 22:58:21 dist-test-slave-ncp9 krb5kdc[18574](info): commencing operation
krb5kdc: starting...
W20260809 22:58:22.925576 15814 mini_kdc.cc:121] Time spent starting KDC: real 1.205s user 0.008s sys 0.038s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786315935617919-15814-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786315935617919-15814-0/krb5kdc/test-user.keytab.
Aug 09 22:58:23 dist-test-slave-ncp9 krb5kdc[18574](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316303, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T22:58:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T22:58:23Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.113.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.113.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.113.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.190.keytab.
Entry for principal kudu/127.15.113.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.190.keytab.
I20260809 22:58:23.223173 15814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskItYPJR/build/asan/bin/kudu
/tmp/dist-test-taskItYPJR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786315935617919-15814-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786315935617919-15814-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786315935617919-15814-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786315935617919-15814-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.113.190:42379
--webserver_interface=127.15.113.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.190.keytab
--principal=kudu/127.15.113.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.113.148:35863
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.113.190:42379,127.15.113.189:38973,127.15.113.188:40915
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786315935617919-15814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786315935617919-15814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786315935617919-15814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:58:23.549858 18592 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260809 22:58:23.550180 18592 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:58:23.550282 18592 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:58:23.550385 18592 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:58:23.557359 18592 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 22:58:23.557435 18592 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:58:23.557518 18592 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:58:23.557579 18592 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 22:58:23.557669 18592 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 22:58:23.567024 18592 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.113.148:35863
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786315935617919-15814-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786315935617919-15814-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.113.190:42379,127.15.113.189:38973,127.15.113.188:40915
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.190.keytab
--principal=kudu/127.15.113.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.113.190:42379
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786315935617919-15814-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.113.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786315935617919-15814-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:58:23.568228 18592 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:58:23.569566 18592 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:58:23.580598 18603 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:58:23.581000 18600 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:58:23.580708 18606 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:58:23.581211 18592 server_base.cc:1034] running on GCE node
I20260809 22:58:23.582041 18592 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:58:23.583355 18592 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:58:23.584570 18592 hybrid_clock.cc:648] HybridClock initialized: now 1786316303584489 us; error 51 us; skew 500 ppm
I20260809 22:58:23.584877 18592 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:58:23 dist-test-slave-ncp9 krb5kdc[18574](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316303, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.113.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:58:23.591595 18592 init.cc:377] Logged in from keytab as kudu/127.15.113.190@KRBTEST.COM (short username kudu)
I20260809 22:58:23.592962 18592 webserver.cc:460] Webserver started at http://127.15.113.190:41605/ using document root <none> and password file <none>
I20260809 22:58:23.593644 18592 fs_manager.cc:359] Metadata directory not provided
I20260809 22:58:23.593750 18592 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:58:23.594053 18592 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:58:23.596966 18592 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786315935617919-15814-0/minicluster-data/master-0/data/instance:
uuid: "8f81ddb676094a5b9e8f78b0d6e7ee3b"
format_stamp: "Formatted at 2026-08-09 22:58:23 on dist-test-slave-ncp9"
server_key: "9ff8254a31055c41f0ce3918ab640153"
server_key_iv: "91aa63a1afb76fc80d97fb0c2929870a"
server_key_version: "encryptionkey@0"
I20260809 22:58:23.597754 18592 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786315935617919-15814-0/minicluster-data/master-0/wal/instance:
uuid: "8f81ddb676094a5b9e8f78b0d6e7ee3b"
format_stamp: "Formatted at 2026-08-09 22:58:23 on dist-test-slave-ncp9"
server_key: "9ff8254a31055c41f0ce3918ab640153"
server_key_iv: "91aa63a1afb76fc80d97fb0c2929870a"
server_key_version: "encryptionkey@0"
I20260809 22:58:23.602648 18592 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260809 22:58:23.605999 18617 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:58:23.607208 18592 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 22:58:23.607360 18592 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786315935617919-15814-0/minicluster-data/master-0/data,/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786315935617919-15814-0/minicluster-data/master-0/wal
uuid: "8f81ddb676094a5b9e8f78b0d6e7ee3b"
format_stamp: "Formatted at 2026-08-09 22:58:23 on dist-test-slave-ncp9"
server_key: "9ff8254a31055c41f0ce3918ab640153"
server_key_iv: "91aa63a1afb76fc80d97fb0c2929870a"
server_key_version: "encryptionkey@0"
I20260809 22:58:23.607547 18592 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786315935617919-15814-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786315935617919-15814-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786315935617919-15814-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:58:23.631784 18592 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:58:23.635641 18592 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:58:23.635893 18592 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:58:23.653368 18592 rpc_server.cc:307] RPC server started. Bound to: 127.15.113.190:42379
I20260809 22:58:23.653414 18686 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.113.190:42379 every 8 connection(s)
I20260809 22:58:23.654688 18592 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786315935617919-15814-0/minicluster-data/master-0/data/info.pb
I20260809 22:58:23.658435 18688 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:58:23.659154 15814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskItYPJR/build/asan/bin/kudu as pid 18592
I20260809 22:58:23.659337 15814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786315935617919-15814-0/minicluster-data/master-0/wal/instance
I20260809 22:58:23.659785 15814 external_mini_cluster.cc:1428] Setting key b5d20f601b2f766bdae41332814e2b79
I20260809 22:58:23.664788 18688 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.113.190" port: 42379 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 22:58:23 dist-test-slave-ncp9 krb5kdc[18574](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316303, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.113.190@KRBTEST.COM for kudu/127.15.113.190@KRBTEST.COM
I20260809 22:58:23.686967 18690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:58:23.667867 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:42379 (local address 127.0.0.1:46754)
0809 22:58:23.668747 (+ 880us) negotiation.cc:107] Waiting for socket to connect
0809 22:58:23.668810 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0809 22:58:23.670638 (+ 1828us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:58:23.671526 (+ 888us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:58:23.671531 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:58:23.672077 (+ 546us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:58:23.672411 (+ 334us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:58:23.672427 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:58:23.674431 (+ 2004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:58:23.674435 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:58:23.675467 (+ 1032us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:58:23.675478 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:58:23.676354 (+ 876us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:58:23.676360 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:58:23.676564 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:58:23.677232 (+ 668us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:58:23.677254 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:58:23.679459 (+ 2205us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:58:23.683258 (+ 3799us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:58:23.683269 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:58:23.683277 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:58:23.683647 (+ 370us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:58:23.684128 (+ 481us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:58:23.684131 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:58:23.684134 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:58:23.684281 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:58:23.685118 (+ 837us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:58:23.685124 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:58:23.685304 (+ 180us) client_negotiation.cc:769] Sending connection context
0809 22:58:23.685501 (+ 197us) client_negotiation.cc:240] Negotiation successful
0809 22:58:23.685858 (+ 357us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":580,"mutex_wait_us":33,"thread_start_us":362,"threads_started":1}
I20260809 22:58:23.686995 18691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:58:23.667861 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46754 (local address 127.15.113.190:42379)
0809 22:58:23.669434 (+ 1573us) server_negotiation.cc:206] Beginning negotiation
0809 22:58:23.669442 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 22:58:23.669500 (+ 58us) server_negotiation.cc:407] Connection header received
0809 22:58:23.671019 (+ 1519us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:58:23.671026 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:58:23.671317 (+ 291us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:58:23.671515 (+ 198us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:58:23.672614 (+ 1099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:58:23.674294 (+ 1680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:58:23.675598 (+ 1304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:58:23.676120 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:58:23.676302 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:58:23.679684 (+ 3382us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:58:23.679694 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:58:23.679708 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:58:23.679747 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:58:23.683121 (+ 3374us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:58:23.683783 (+ 662us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:58:23.683792 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:58:23.683796 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:58:23.684021 (+ 225us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:58:23.684379 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:58:23.684383 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:58:23.684386 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:58:23.684996 (+ 610us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:58:23.685194 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0809 22:58:23.685667 (+ 473us) server_negotiation.cc:299] Negotiation successful
0809 22:58:23.685995 (+ 328us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":89,"server-negotiator.queue_time_us":1379,"spinlock_wait_cycles":68992,"thread_start_us":1198,"threads_started":1}
I20260809 22:58:23.689850 18688 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.113.189" port: 38973 } has no permanent_uuid. Determining permanent_uuid...
I20260809 22:58:23.690922 18690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:58:23.690452 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.189:38973 (local address 127.0.0.1:44656)
0809 22:58:23.690647 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0809 22:58:23.690781 (+ 134us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.113.189:38973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
W20260809 22:58:23.691196 18620 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.15.113.189:38973: connect: Connection refused (error 111)
I20260809 22:58:23.692755 18690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:58:23.692336 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.189:38973 (local address 127.0.0.1:44664)
0809 22:58:23.692504 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0809 22:58:23.692591 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.113.189:38973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260809 22:58:23.692976 18688 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.113.189:38973: Network error: Client connection negotiation failed: client connection to 127.15.113.189:38973: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.15.113.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.113.189@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.113.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.189.keytab.
Entry for principal kudu/127.15.113.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.189.keytab.
I20260809 22:58:23.733654 15814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskItYPJR/build/asan/bin/kudu
/tmp/dist-test-taskItYPJR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786315935617919-15814-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786315935617919-15814-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786315935617919-15814-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786315935617919-15814-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.113.189:38973
--webserver_interface=127.15.113.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.189.keytab
--principal=kudu/127.15.113.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.113.148:35863
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.113.190:42379,127.15.113.189:38973,127.15.113.188:40915
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786315935617919-15814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786315935617919-15814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786315935617919-15814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 22:58:23.742309 18688 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.113.189" port: 38973 } attempt: 1
I20260809 22:58:23.743394 18690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:58:23.742941 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.189:38973 (local address 127.0.0.1:44672)
0809 22:58:23.743136 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0809 22:58:23.743266 (+ 130us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.113.189:38973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
I20260809 22:58:23.744366 18690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:58:23.744036 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.189:38973 (local address 127.0.0.1:44674)
0809 22:58:23.744179 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0809 22:58:23.744244 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.113.189:38973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260809 22:58:23.744545 18688 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.113.189:38973: Network error: Client connection negotiation failed: client connection to 127.15.113.189:38973: connect: Connection refused (error 111)
I20260809 22:58:23.812780 18688 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.113.189" port: 38973 } attempt: 2
I20260809 22:58:23.814030 18690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:58:23.813455 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.189:38973 (local address 127.0.0.1:44686)
0809 22:58:23.813724 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0809 22:58:23.813844 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.113.189:38973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
I20260809 22:58:23.815116 18690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:58:23.814685 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.189:38973 (local address 127.0.0.1:44696)
0809 22:58:23.814854 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0809 22:58:23.814953 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.113.189:38973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260809 22:58:23.815320 18688 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.113.189:38973: Network error: Client connection negotiation failed: client connection to 127.15.113.189:38973: connect: Connection refused (error 111)
I20260809 22:58:23.906587 18688 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.113.189" port: 38973 } attempt: 3
I20260809 22:58:23.907804 18690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:58:23.907291 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.189:38973 (local address 127.0.0.1:44698)
0809 22:58:23.907516 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0809 22:58:23.907637 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.113.189:38973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
I20260809 22:58:23.908872 18690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:58:23.908474 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.189:38973 (local address 127.0.0.1:44706)
0809 22:58:23.908621 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0809 22:58:23.908707 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.113.189:38973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260809 22:58:23.909106 18688 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.113.189:38973: Network error: Client connection negotiation failed: client connection to 127.15.113.189:38973: connect: Connection refused (error 111)
I20260809 22:58:24.052402 18688 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.113.189" port: 38973 } attempt: 4
I20260809 22:58:24.053743 18690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:58:24.053093 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.189:38973 (local address 127.0.0.1:44712)
0809 22:58:24.053454 (+ 361us) negotiation.cc:107] Waiting for socket to connect
0809 22:58:24.053578 (+ 124us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.113.189:38973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":176}
I20260809 22:58:24.054744 18690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:58:24.054353 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.189:38973 (local address 127.0.0.1:44716)
0809 22:58:24.054504 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0809 22:58:24.054597 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.113.189:38973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260809 22:58:24.054924 18688 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.113.189:38973: Network error: Client connection negotiation failed: client connection to 127.15.113.189:38973: connect: Connection refused (error 111)
W20260809 22:58:24.067420 18699 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260809 22:58:24.067687 18699 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:58:24.067754 18699 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:58:24.067831 18699 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:58:24.074755 18699 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 22:58:24.074828 18699 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:58:24.074890 18699 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:58:24.074939 18699 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 22:58:24.075004 18699 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 22:58:24.084564 18699 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.113.148:35863
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786315935617919-15814-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786315935617919-15814-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.15.113.190:42379,127.15.113.189:38973,127.15.113.188:40915
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.189.keytab
--principal=kudu/127.15.113.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.113.189:38973
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786315935617919-15814-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.15.113.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786315935617919-15814-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:58:24.085640 18699 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:58:24.086838 18699 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:58:24.098201 18708 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:58:24.098332 18712 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:58:24.098444 18709 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:58:24.099269 18699 server_base.cc:1034] running on GCE node
I20260809 22:58:24.099946 18699 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:58:24.101227 18699 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:58:24.102432 18699 hybrid_clock.cc:648] HybridClock initialized: now 1786316304102371 us; error 38 us; skew 500 ppm
I20260809 22:58:24.102732 18699 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:58:24 dist-test-slave-ncp9 krb5kdc[18574](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316304, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.113.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:58:24.105767 18699 init.cc:377] Logged in from keytab as kudu/127.15.113.189@KRBTEST.COM (short username kudu)
I20260809 22:58:24.107066 18699 webserver.cc:460] Webserver started at http://127.15.113.189:38245/ using document root <none> and password file <none>
I20260809 22:58:24.107698 18699 fs_manager.cc:359] Metadata directory not provided
I20260809 22:58:24.107797 18699 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:58:24.108085 18699 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:58:24.110939 18699 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786315935617919-15814-0/minicluster-data/master-1/data/instance:
uuid: "d51e830c048242209eb6513f255daa90"
format_stamp: "Formatted at 2026-08-09 22:58:24 on dist-test-slave-ncp9"
server_key: "1c5f40a7d749dec4890a72906e00fd07"
server_key_iv: "40695d094440610774df84c076cf6f49"
server_key_version: "encryptionkey@0"
I20260809 22:58:24.111690 18699 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786315935617919-15814-0/minicluster-data/master-1/wal/instance:
uuid: "d51e830c048242209eb6513f255daa90"
format_stamp: "Formatted at 2026-08-09 22:58:24 on dist-test-slave-ncp9"
server_key: "1c5f40a7d749dec4890a72906e00fd07"
server_key_iv: "40695d094440610774df84c076cf6f49"
server_key_version: "encryptionkey@0"
I20260809 22:58:24.116336 18699 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260809 22:58:24.119544 18728 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:58:24.120767 18699 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 22:58:24.120947 18699 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786315935617919-15814-0/minicluster-data/master-1/data,/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786315935617919-15814-0/minicluster-data/master-1/wal
uuid: "d51e830c048242209eb6513f255daa90"
format_stamp: "Formatted at 2026-08-09 22:58:24 on dist-test-slave-ncp9"
server_key: "1c5f40a7d749dec4890a72906e00fd07"
server_key_iv: "40695d094440610774df84c076cf6f49"
server_key_version: "encryptionkey@0"
I20260809 22:58:24.121135 18699 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786315935617919-15814-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786315935617919-15814-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786315935617919-15814-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:58:24.148406 18699 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:58:24.152098 18699 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:58:24.152334 18699 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:58:24.169121 18699 rpc_server.cc:307] RPC server started. Bound to: 127.15.113.189:38973
I20260809 22:58:24.169142 18804 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.113.189:38973 every 8 connection(s)
I20260809 22:58:24.170351 18699 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786315935617919-15814-0/minicluster-data/master-1/data/info.pb
I20260809 22:58:24.172350 15814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskItYPJR/build/asan/bin/kudu as pid 18699
I20260809 22:58:24.172544 15814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786315935617919-15814-0/minicluster-data/master-1/wal/instance
I20260809 22:58:24.173146 15814 external_mini_cluster.cc:1428] Setting key 36756a8dfd63f4eea32058ba442ad72d
I20260809 22:58:24.174536 18805 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:58:24.182844 18805 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.113.190" port: 42379 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 22:58:24 dist-test-slave-ncp9 krb5kdc[18574](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316304, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.113.189@KRBTEST.COM for kudu/127.15.113.190@KRBTEST.COM
I20260809 22:58:24.203470 18691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:58:24.185782 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46764 (local address 127.15.113.190:42379)
0809 22:58:24.185989 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0809 22:58:24.185995 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:58:24.187158 (+ 1163us) server_negotiation.cc:407] Connection header received
0809 22:58:24.188972 (+ 1814us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:58:24.188978 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:58:24.189058 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:58:24.189154 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:58:24.190984 (+ 1830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:58:24.191646 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:58:24.193681 (+ 2035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:58:24.194081 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:58:24.194186 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:58:24.197888 (+ 3702us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:58:24.197894 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:58:24.197897 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:58:24.197929 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:58:24.200496 (+ 2567us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:58:24.201255 (+ 759us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:58:24.201261 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:58:24.201263 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:58:24.201383 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:58:24.201790 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:58:24.201793 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:58:24.201795 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:58:24.202098 (+ 303us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:58:24.202261 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0809 22:58:24.202920 (+ 659us) server_negotiation.cc:299] Negotiation successful
0809 22:58:24.203164 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260809 22:58:24.204304 18807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:58:24.186246 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:42379 (local address 127.0.0.1:46764)
0809 22:58:24.186971 (+ 725us) negotiation.cc:107] Waiting for socket to connect
0809 22:58:24.187023 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0809 22:58:24.188691 (+ 1668us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:58:24.189416 (+ 725us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:58:24.189426 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:58:24.190116 (+ 690us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:58:24.190831 (+ 715us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:58:24.190854 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:58:24.191798 (+ 944us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:58:24.191803 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:58:24.193561 (+ 1758us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:58:24.193576 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:58:24.194313 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:58:24.194319 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:58:24.194581 (+ 262us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:58:24.195245 (+ 664us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:58:24.195292 (+ 47us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:58:24.197674 (+ 2382us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:58:24.200641 (+ 2967us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:58:24.200650 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:58:24.200664 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:58:24.201135 (+ 471us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:58:24.201488 (+ 353us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:58:24.201493 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:58:24.201496 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:58:24.201673 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:58:24.202244 (+ 571us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:58:24.202252 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:58:24.202554 (+ 302us) client_negotiation.cc:769] Sending connection context
0809 22:58:24.202791 (+ 237us) client_negotiation.cc:240] Negotiation successful
0809 22:58:24.203255 (+ 464us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":482,"thread_start_us":311,"threads_started":1}
I20260809 22:58:24.206009 18805 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.113.189" port: 38973 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.15.113.188@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.113.188@KRBTEST.COM" created.
Aug 09 22:58:24 dist-test-slave-ncp9 krb5kdc[18574](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316304, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.113.189@KRBTEST.COM for kudu/127.15.113.189@KRBTEST.COM
I20260809 22:58:24.220252 18807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:58:24.206721 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.189:38973 (local address 127.0.0.1:44726)
0809 22:58:24.206901 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0809 22:58:24.206922 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0809 22:58:24.207019 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:58:24.207945 (+ 926us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:58:24.207949 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:58:24.208302 (+ 353us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:58:24.208427 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:58:24.208435 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:58:24.209864 (+ 1429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:58:24.209870 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:58:24.210824 (+ 954us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:58:24.210836 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:58:24.211944 (+ 1108us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:58:24.211948 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:58:24.212068 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:58:24.212578 (+ 510us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:58:24.212594 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:58:24.214554 (+ 1960us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:58:24.218005 (+ 3451us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:58:24.218009 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:58:24.218012 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:58:24.218352 (+ 340us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:58:24.218708 (+ 356us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:58:24.218711 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:58:24.218713 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:58:24.218848 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:58:24.219480 (+ 632us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:58:24.219485 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:58:24.219610 (+ 125us) client_negotiation.cc:769] Sending connection context
0809 22:58:24.219739 (+ 129us) client_negotiation.cc:240] Negotiation successful
0809 22:58:24.219955 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":63}
I20260809 22:58:24.220506 18809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:58:24.206933 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44726 (local address 127.15.113.189:38973)
0809 22:58:24.207559 (+ 626us) server_negotiation.cc:206] Beginning negotiation
0809 22:58:24.207564 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:58:24.207589 (+ 25us) server_negotiation.cc:407] Connection header received
0809 22:58:24.207640 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:58:24.207645 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:58:24.207804 (+ 159us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:58:24.207910 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:58:24.208579 (+ 669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:58:24.209727 (+ 1148us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:58:24.210953 (+ 1226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:58:24.211804 (+ 851us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:58:24.211920 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:58:24.214756 (+ 2836us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:58:24.214768 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:58:24.214783 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:58:24.214821 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:58:24.217889 (+ 3068us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:58:24.218472 (+ 583us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:58:24.218479 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:58:24.218482 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:58:24.218613 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:58:24.218942 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:58:24.218946 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:58:24.218948 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:58:24.219357 (+ 409us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:58:24.219551 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0809 22:58:24.219897 (+ 346us) server_negotiation.cc:299] Negotiation successful
0809 22:58:24.220133 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":482,"thread_start_us":349,"threads_started":1}
I20260809 22:58:24.223089 18805 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.113.188" port: 40915 } has no permanent_uuid. Determining permanent_uuid...
I20260809 22:58:24.224313 18807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:58:24.223816 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.188:40915 (local address 127.0.0.1:48752)
0809 22:58:24.224046 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0809 22:58:24.224185 (+ 139us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.113.188:40915: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127}
W20260809 22:58:24.224553 18732 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.15.113.188:40915: connect: Connection refused (error 111)
I20260809 22:58:24.226063 18807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:58:24.225730 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.188:40915 (local address 127.0.0.1:48760)
0809 22:58:24.225865 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0809 22:58:24.225951 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.113.188:40915: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260809 22:58:24.226260 18805 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.113.188:40915: Network error: Client connection negotiation failed: client connection to 127.15.113.188:40915: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.113.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.188.keytab.
Entry for principal kudu/127.15.113.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.188.keytab.
I20260809 22:58:24.246066 15814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskItYPJR/build/asan/bin/kudu
/tmp/dist-test-taskItYPJR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786315935617919-15814-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786315935617919-15814-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786315935617919-15814-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786315935617919-15814-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.113.188:40915
--webserver_interface=127.15.113.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.188.keytab
--principal=kudu/127.15.113.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.113.148:35863
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.113.190:42379,127.15.113.189:38973,127.15.113.188:40915
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786315935617919-15814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786315935617919-15814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786315935617919-15814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 22:58:24.275602 18805 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.113.188" port: 40915 } attempt: 1
I20260809 22:58:24.276806 18807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:58:24.276278 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.188:40915 (local address 127.0.0.1:48766)
0809 22:58:24.276494 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0809 22:58:24.276619 (+ 125us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.113.188:40915: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
I20260809 22:58:24.277798 18807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:58:24.277426 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.188:40915 (local address 127.0.0.1:48772)
0809 22:58:24.277576 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0809 22:58:24.277674 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.113.188:40915: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260809 22:58:24.277997 18805 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.113.188:40915: Network error: Client connection negotiation failed: client connection to 127.15.113.188:40915: connect: Connection refused (error 111)
I20260809 22:58:24.346230 18805 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.113.188" port: 40915 } attempt: 2
I20260809 22:58:24.347491 18807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:58:24.346929 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.188:40915 (local address 127.0.0.1:48784)
0809 22:58:24.347164 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0809 22:58:24.347286 (+ 122us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.113.188:40915: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":79}
I20260809 22:58:24.348508 18807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:58:24.348144 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.188:40915 (local address 127.0.0.1:48796)
0809 22:58:24.348290 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0809 22:58:24.348375 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.113.188:40915: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260809 22:58:24.348740 18805 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.113.188:40915: Network error: Client connection negotiation failed: client connection to 127.15.113.188:40915: connect: Connection refused (error 111)
I20260809 22:58:24.354152 18688 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.113.189" port: 38973 } attempt: 5
Aug 09 22:58:24 dist-test-slave-ncp9 krb5kdc[18574](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316303, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.113.190@KRBTEST.COM for kudu/127.15.113.189@KRBTEST.COM
I20260809 22:58:24.367787 18809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:58:24.354885 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44734 (local address 127.15.113.189:38973)
0809 22:58:24.355061 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0809 22:58:24.355066 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:58:24.355085 (+ 19us) server_negotiation.cc:407] Connection header received
0809 22:58:24.355276 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:58:24.355280 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:58:24.355352 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:58:24.355433 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:58:24.356427 (+ 994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:58:24.357190 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:58:24.358822 (+ 1632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:58:24.359338 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:58:24.359436 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:58:24.362826 (+ 3390us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:58:24.362831 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:58:24.362834 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:58:24.362865 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:58:24.365357 (+ 2492us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:58:24.365973 (+ 616us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:58:24.365977 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:58:24.365980 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:58:24.366096 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:58:24.366452 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:58:24.366457 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:58:24.366460 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:58:24.366738 (+ 278us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:58:24.366871 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0809 22:58:24.367237 (+ 366us) server_negotiation.cc:299] Negotiation successful
0809 22:58:24.367470 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260809 22:58:24.367956 18690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:58:24.354814 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.189:38973 (local address 127.0.0.1:44734)
0809 22:58:24.355010 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0809 22:58:24.355029 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 22:58:24.355161 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:58:24.355502 (+ 341us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:58:24.355507 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:58:24.356035 (+ 528us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:58:24.356276 (+ 241us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:58:24.356289 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:58:24.357316 (+ 1027us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:58:24.357322 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:58:24.358627 (+ 1305us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:58:24.358667 (+ 40us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:58:24.359494 (+ 827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:58:24.359501 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:58:24.359736 (+ 235us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:58:24.360353 (+ 617us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:58:24.360375 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:58:24.362648 (+ 2273us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:58:24.365489 (+ 2841us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:58:24.365495 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:58:24.365499 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:58:24.365861 (+ 362us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:58:24.366200 (+ 339us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:58:24.366206 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:58:24.366209 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:58:24.366349 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:58:24.366851 (+ 502us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:58:24.366856 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:58:24.366999 (+ 143us) client_negotiation.cc:769] Sending connection context
0809 22:58:24.367131 (+ 132us) client_negotiation.cc:240] Negotiation successful
0809 22:58:24.367475 (+ 344us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":62}
I20260809 22:58:24.370132 18688 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.113.188" port: 40915 } has no permanent_uuid. Determining permanent_uuid...
I20260809 22:58:24.371352 18690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:58:24.370915 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.188:40915 (local address 127.0.0.1:48798)
0809 22:58:24.371102 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0809 22:58:24.371200 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.113.188:40915: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260809 22:58:24.372442 18690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:58:24.372009 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.188:40915 (local address 127.0.0.1:48802)
0809 22:58:24.372224 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0809 22:58:24.372306 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.113.188:40915: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114}
W20260809 22:58:24.372640 18688 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.113.188:40915: Network error: Client connection negotiation failed: client connection to 127.15.113.188:40915: connect: Connection refused (error 111)
I20260809 22:58:24.423853 18688 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.113.188" port: 40915 } attempt: 1
I20260809 22:58:24.424940 18690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:58:24.424455 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.188:40915 (local address 127.0.0.1:48808)
0809 22:58:24.424654 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0809 22:58:24.424746 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.113.188:40915: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260809 22:58:24.425952 18690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:58:24.425643 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.188:40915 (local address 127.0.0.1:48814)
0809 22:58:24.425784 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0809 22:58:24.425853 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.113.188:40915: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260809 22:58:24.426153 18688 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.113.188:40915: Network error: Client connection negotiation failed: client connection to 127.15.113.188:40915: connect: Connection refused (error 111)
I20260809 22:58:24.439960 18805 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.113.188" port: 40915 } attempt: 3
I20260809 22:58:24.441134 18807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:58:24.440577 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.188:40915 (local address 127.0.0.1:48816)
0809 22:58:24.440796 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0809 22:58:24.440934 (+ 138us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.113.188:40915: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
I20260809 22:58:24.442231 18807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:58:24.441743 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.188:40915 (local address 127.0.0.1:48832)
0809 22:58:24.441976 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0809 22:58:24.442073 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.113.188:40915: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130}
W20260809 22:58:24.442472 18805 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.113.188:40915: Network error: Client connection negotiation failed: client connection to 127.15.113.188:40915: connect: Connection refused (error 111)
I20260809 22:58:24.494341 18688 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.113.188" port: 40915 } attempt: 2
I20260809 22:58:24.495409 18690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:58:24.494953 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.188:40915 (local address 127.0.0.1:48842)
0809 22:58:24.495146 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0809 22:58:24.495250 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.113.188:40915: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260809 22:58:24.496411 18690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:58:24.496127 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.188:40915 (local address 127.0.0.1:48854)
0809 22:58:24.496244 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0809 22:58:24.496312 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.113.188:40915: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260809 22:58:24.496613 18688 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.113.188:40915: Network error: Client connection negotiation failed: client connection to 127.15.113.188:40915: connect: Connection refused (error 111)
I20260809 22:58:24.585693 18805 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.113.188" port: 40915 } attempt: 4
I20260809 22:58:24.586884 18807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:58:24.586333 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.188:40915 (local address 127.0.0.1:48864)
0809 22:58:24.586572 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0809 22:58:24.586689 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.113.188:40915: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
I20260809 22:58:24.587838 18807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:58:24.587489 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.188:40915 (local address 127.0.0.1:48866)
0809 22:58:24.587625 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0809 22:58:24.587717 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.113.188:40915: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260809 22:58:24.588056 18805 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.113.188:40915: Network error: Client connection negotiation failed: client connection to 127.15.113.188:40915: connect: Connection refused (error 111)
W20260809 22:58:24.590372 18812 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260809 22:58:24.590687 18812 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:58:24.590775 18812 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:58:24.590886 18812 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:58:24.598047 18812 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 22:58:24.598131 18812 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:58:24.598227 18812 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:58:24.598289 18812 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 22:58:24.598372 18812 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 22:58:24.602802 18688 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.113.188" port: 40915 } attempt: 3
I20260809 22:58:24.603858 18690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:58:24.603402 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.188:40915 (local address 127.0.0.1:48874)
0809 22:58:24.603616 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0809 22:58:24.603710 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.113.188:40915: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260809 22:58:24.604785 18690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:58:24.604442 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.188:40915 (local address 127.0.0.1:48888)
0809 22:58:24.604609 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0809 22:58:24.604692 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.113.188:40915: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":87}
W20260809 22:58:24.604990 18688 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.113.188:40915: Network error: Client connection negotiation failed: client connection to 127.15.113.188:40915: connect: Connection refused (error 111)
I20260809 22:58:24.608666 18812 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.113.148:35863
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786315935617919-15814-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786315935617919-15814-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.15.113.190:42379,127.15.113.189:38973,127.15.113.188:40915
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786315935617919-15814-0/krb5kdc/kudu_127.15.113.188.keytab
--principal=kudu/127.15.113.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.113.188:40915
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786315935617919-15814-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.15.113.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786315935617919-15814-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:58:24.609963 18812 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:58:24.611325 18812 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:58:24.622747 18820 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:58:24.622766 18825 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:58:24.622720 18821 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:58:24.623535 18812 server_base.cc:1034] running on GCE node
I20260809 22:58:24.624291 18812 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:58:24.625692 18812 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:58:24.626912 18812 hybrid_clock.cc:648] HybridClock initialized: now 1786316304626835 us; error 43 us; skew 500 ppm
I20260809 22:58:24.627220 18812 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:58:24 dist-test-slave-ncp9 krb5kdc[18574](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316304, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.113.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:58:24.630177 18812 init.cc:377] Logged in from keytab as kudu/127.15.113.188@KRBTEST.COM (short username kudu)
I20260809 22:58:24.631541 18812 webserver.cc:460] Webserver started at http://127.15.113.188:46771/ using document root <none> and password file <none>
I20260809 22:58:24.632205 18812 fs_manager.cc:359] Metadata directory not provided
I20260809 22:58:24.632302 18812 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:58:24.632597 18812 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:58:24.635468 18812 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786315935617919-15814-0/minicluster-data/master-2/data/instance:
uuid: "cfe847d21d174998bd4a854e65a9e8d5"
format_stamp: "Formatted at 2026-08-09 22:58:24 on dist-test-slave-ncp9"
server_key: "09b266b998608de1703873d40b8c5e40"
server_key_iv: "85b031120783fecb4be0795f578c35d5"
server_key_version: "encryptionkey@0"
I20260809 22:58:24.636245 18812 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786315935617919-15814-0/minicluster-data/master-2/wal/instance:
uuid: "cfe847d21d174998bd4a854e65a9e8d5"
format_stamp: "Formatted at 2026-08-09 22:58:24 on dist-test-slave-ncp9"
server_key: "09b266b998608de1703873d40b8c5e40"
server_key_iv: "85b031120783fecb4be0795f578c35d5"
server_key_version: "encryptionkey@0"
I20260809 22:58:24.641178 18812 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260809 22:58:24.644383 18835 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:58:24.645635 18812 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 22:58:24.645790 18812 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786315935617919-15814-0/minicluster-data/master-2/data,/tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786315935617919-15814-0/minicluster-data/master-2/wal
uuid: "cfe847d21d174998bd4a854e65a9e8d5"
format_stamp: "Formatted at 2026-08-09 22:58:24 on dist-test-slave-ncp9"
server_key: "09b266b998608de1703873d40b8c5e40"
server_key_iv: "85b031120783fecb4be0795f578c35d5"
server_key_version: "encryptionkey@0"
I20260809 22:58:24.645989 18812 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786315935617919-15814-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786315935617919-15814-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786315935617919-15814-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:58:24.674136 18812 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:58:24.677763 18812 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:58:24.677994 18812 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:58:24.695128 18812 rpc_server.cc:307] RPC server started. Bound to: 127.15.113.188:40915
I20260809 22:58:24.695137 18917 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.113.188:40915 every 8 connection(s)
I20260809 22:58:24.696431 18812 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786315935617919-15814-0/minicluster-data/master-2/data/info.pb
I20260809 22:58:24.700102 18918 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:58:24.703910 15814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskItYPJR/build/asan/bin/kudu as pid 18812
I20260809 22:58:24.704099 15814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskItYPJR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786315935617919-15814-0/minicluster-data/master-2/wal/instance
I20260809 22:58:24.704675 15814 external_mini_cluster.cc:1428] Setting key 23984c93b24aa7cb5a1259fe21a6746a
I20260809 22:58:24.707729 18918 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.113.190" port: 42379 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 22:58:24 dist-test-slave-ncp9 krb5kdc[18574](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316303, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.113.188@KRBTEST.COM
Aug 09 22:58:24 dist-test-slave-ncp9 krb5kdc[18574](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316304, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.113.188@KRBTEST.COM for kudu/127.15.113.190@KRBTEST.COM
I20260809 22:58:24.726392 18928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:58:24.711613 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46770 (local address 127.15.113.190:42379)
0809 22:58:24.712220 (+ 607us) server_negotiation.cc:206] Beginning negotiation
0809 22:58:24.712225 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:58:24.712757 (+ 532us) server_negotiation.cc:407] Connection header received
0809 22:58:24.712943 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:58:24.712950 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:58:24.713013 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:58:24.713100 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:58:24.714365 (+ 1265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:58:24.715315 (+ 950us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:58:24.716875 (+ 1560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:58:24.717532 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:58:24.717664 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:58:24.720850 (+ 3186us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:58:24.720856 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:58:24.720861 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:58:24.720922 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:58:24.723427 (+ 2505us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:58:24.724075 (+ 648us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:58:24.724081 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:58:24.724084 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:58:24.724236 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:58:24.724605 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:58:24.724609 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:58:24.724612 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:58:24.724945 (+ 333us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:58:24.725090 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0809 22:58:24.725800 (+ 710us) server_negotiation.cc:299] Negotiation successful
0809 22:58:24.726004 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":475,"thread_start_us":328,"threads_started":1}
I20260809 22:58:24.727155 18923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:58:24.706564 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48892 (local address 127.15.113.188:40915)
0809 22:58:24.707266 (+ 702us) server_negotiation.cc:206] Beginning negotiation
0809 22:58:24.707277 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 22:58:24.707316 (+ 39us) server_negotiation.cc:407] Connection header received
0809 22:58:24.708864 (+ 1548us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:58:24.708886 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:58:24.709344 (+ 458us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:58:24.709702 (+ 358us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:58:24.711490 (+ 1788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:58:24.713540 (+ 2050us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:58:24.715003 (+ 1463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:58:24.715837 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:58:24.716014 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:58:24.720362 (+ 4348us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:58:24.720370 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:58:24.720382 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:58:24.720411 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:58:24.722939 (+ 2528us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:58:24.723632 (+ 693us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:58:24.723640 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:58:24.723642 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:58:24.723799 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:58:24.724172 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:58:24.724176 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:58:24.724179 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:58:24.724881 (+ 702us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:58:24.725227 (+ 346us) server_negotiation.cc:1035] Waiting for connection context
0809 22:58:24.725586 (+ 359us) server_negotiation.cc:299] Negotiation successful
0809 22:58:24.726017 (+ 431us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":517,"thread_start_us":316,"threads_started":1}
I20260809 22:58:24.727176 18929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:58:24.711647 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.190:42379 (local address 127.0.0.1:46770)
0809 22:58:24.712604 (+ 957us) negotiation.cc:107] Waiting for socket to connect
0809 22:58:24.712656 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0809 22:58:24.712809 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:58:24.713200 (+ 391us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:58:24.713208 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:58:24.713959 (+ 751us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:58:24.714213 (+ 254us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:58:24.714222 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:58:24.715464 (+ 1242us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:58:24.715469 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:58:24.716731 (+ 1262us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:58:24.716743 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:58:24.717726 (+ 983us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:58:24.717731 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:58:24.717848 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:58:24.718415 (+ 567us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:58:24.718441 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:58:24.720675 (+ 2234us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:58:24.723549 (+ 2874us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:58:24.723560 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:58:24.723571 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:58:24.723966 (+ 395us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:58:24.724364 (+ 398us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:58:24.724370 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:58:24.724374 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:58:24.724499 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:58:24.725088 (+ 589us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:58:24.725095 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:58:24.725362 (+ 267us) client_negotiation.cc:769] Sending connection context
0809 22:58:24.725680 (+ 318us) client_negotiation.cc:240] Negotiation successful
0809 22:58:24.726153 (+ 473us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":702,"mutex_wait_us":155,"thread_start_us":538,"threads_started":1}
I20260809 22:58:24.729473 18918 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.113.189" port: 38973 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 22:58:24 dist-test-slave-ncp9 krb5kdc[18574](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316304, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.113.188@KRBTEST.COM for kudu/127.15.113.189@KRBTEST.COM
Aug 09 22:58:24 dist-test-slave-ncp9 krb5kdc[18574](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316303, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.113.190@KRBTEST.COM
I20260809 22:58:24.742084 18929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:58:24.730216 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.189:38973 (local address 127.0.0.1:44746)
0809 22:58:24.730367 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0809 22:58:24.730380 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 22:58:24.730453 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:58:24.731069 (+ 616us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:58:24.731073 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:58:24.731367 (+ 294us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:58:24.731494 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:58:24.731501 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:58:24.732491 (+ 990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:58:24.732495 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:58:24.733390 (+ 895us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:58:24.733402 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:58:24.734072 (+ 670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:58:24.734078 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:58:24.734251 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:58:24.734722 (+ 471us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:58:24.734751 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:58:24.736605 (+ 1854us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:58:24.739878 (+ 3273us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:58:24.739882 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:58:24.739885 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:58:24.740237 (+ 352us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:58:24.740641 (+ 404us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:58:24.740645 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:58:24.740647 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:58:24.740781 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:58:24.741309 (+ 528us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:58:24.741314 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:58:24.741475 (+ 161us) client_negotiation.cc:769] Sending connection context
0809 22:58:24.741586 (+ 111us) client_negotiation.cc:240] Negotiation successful
0809 22:58:24.741845 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":56}
I20260809 22:58:24.742219 18809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:58:24.730565 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44746 (local address 127.15.113.189:38973)
0809 22:58:24.730767 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0809 22:58:24.730772 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:58:24.730791 (+ 19us) server_negotiation.cc:407] Connection header received
0809 22:58:24.730847 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:58:24.730852 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:58:24.730928 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:58:24.731032 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:58:24.731647 (+ 615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:58:24.732353 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:58:24.733525 (+ 1172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:58:24.733942 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:58:24.734048 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:58:24.736775 (+ 2727us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:58:24.736780 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:58:24.736783 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:58:24.736815 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:58:24.739758 (+ 2943us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:58:24.740349 (+ 591us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:58:24.740354 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:58:24.740358 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:58:24.740521 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:58:24.740885 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:58:24.740909 (+ 24us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:58:24.740912 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:58:24.741178 (+ 266us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:58:24.741295 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0809 22:58:24.741695 (+ 400us) server_negotiation.cc:299] Negotiation successful
0809 22:58:24.741854 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260809 22:58:24.743166 18918 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.113.188" port: 40915 } has no permanent_uuid. Determining permanent_uuid...
I20260809 22:58:24.743516 18928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:58:24.729973 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46782 (local address 127.15.113.190:42379)
0809 22:58:24.730133 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0809 22:58:24.730138 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:58:24.730155 (+ 17us) server_negotiation.cc:407] Connection header received
0809 22:58:24.730282 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:58:24.730286 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:58:24.730370 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:58:24.730467 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:58:24.731814 (+ 1347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:58:24.732472 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:58:24.733669 (+ 1197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:58:24.734077 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:58:24.734179 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:58:24.737996 (+ 3817us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:58:24.738003 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:58:24.738007 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:58:24.738045 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:58:24.740796 (+ 2751us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:58:24.741429 (+ 633us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:58:24.741432 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:58:24.741434 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:58:24.741538 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:58:24.741939 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:58:24.741944 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:58:24.741948 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:58:24.742280 (+ 332us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:58:24.742528 (+ 248us) server_negotiation.cc:1035] Waiting for connection context
0809 22:58:24.743075 (+ 547us) server_negotiation.cc:299] Negotiation successful
0809 22:58:24.743246 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
Aug 09 22:58:24 dist-test-slave-ncp9 krb5kdc[18574](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316304, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.113.188@KRBTEST.COM for kudu/127.15.113.188@KRBTEST.COM
Aug 09 22:58:24 dist-test-slave-ncp9 krb5kdc[18574](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316303, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.113.189@KRBTEST.COM
I20260809 22:58:24.756552 18929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:58:24.743821 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.188:40915 (local address 127.0.0.1:48902)
0809 22:58:24.743961 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0809 22:58:24.743978 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 22:58:24.744051 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:58:24.744320 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:58:24.744323 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:58:24.744682 (+ 359us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:58:24.744808 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:58:24.744816 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:58:24.745883 (+ 1067us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:58:24.745888 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:58:24.747181 (+ 1293us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:58:24.747196 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:58:24.748041 (+ 845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:58:24.748045 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:58:24.748249 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:58:24.748752 (+ 503us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:58:24.748774 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:58:24.751183 (+ 2409us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:58:24.754383 (+ 3200us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:58:24.754388 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:58:24.754393 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:58:24.754733 (+ 340us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:58:24.755055 (+ 322us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:58:24.755060 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:58:24.755064 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:58:24.755185 (+ 121us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:58:24.755652 (+ 467us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:58:24.755657 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:58:24.755795 (+ 138us) client_negotiation.cc:769] Sending connection context
0809 22:58:24.755908 (+ 113us) client_negotiation.cc:240] Negotiation successful
0809 22:58:24.756175 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43,"mutex_wait_us":17}
I20260809 22:58:24.756611 18923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:58:24.743834 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48902 (local address 127.15.113.188:40915)
0809 22:58:24.743988 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0809 22:58:24.743993 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:58:24.744014 (+ 21us) server_negotiation.cc:407] Connection header received
0809 22:58:24.744129 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:58:24.744133 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:58:24.744212 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:58:24.744308 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:58:24.744959 (+ 651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:58:24.745765 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:58:24.747305 (+ 1540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:58:24.747912 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:58:24.748036 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:58:24.751365 (+ 3329us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:58:24.751372 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:58:24.751377 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:58:24.751413 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:58:24.754268 (+ 2855us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:58:24.754835 (+ 567us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:58:24.754840 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:58:24.754844 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:58:24.754956 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:58:24.755281 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:58:24.755286 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:58:24.755290 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:58:24.755538 (+ 248us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:58:24.755663 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0809 22:58:24.756004 (+ 341us) server_negotiation.cc:299] Negotiation successful
0809 22:58:24.756328 (+ 324us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":144,"server-negotiator.queue_time_us":61}
I20260809 22:58:24.760251 18809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:58:24.745345 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44752 (local address 127.15.113.189:38973)
0809 22:58:24.745473 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0809 22:58:24.745476 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 22:58:24.745489 (+ 13us) server_negotiation.cc:407] Connection header received
0809 22:58:24.745522 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:58:24.745525 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:58:24.745571 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:58:24.745654 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:58:24.747230 (+ 1576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:58:24.748209 (+ 979us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:58:24.749960 (+ 1751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:58:24.750657 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:58:24.750777 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:58:24.755164 (+ 4387us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:58:24.755168 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:58:24.755171 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:58:24.755199 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:58:24.757925 (+ 2726us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:58:24.758516 (+ 591us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:58:24.758521 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:58:24.758523 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:58:24.758689 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:58:24.758993 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:58:24.758997 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:58:24.759001 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:58:24.759310 (+ 309us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:58:24.759563 (+ 253us) server_negotiation.cc:1035] Waiting for connection context
0809 22:58:24.759735 (+ 172us) server_negotiation.cc:299] Negotiation successful
0809 22:58:24.759952 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260809 22:58:24.762382 18918 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cfe847d21d174998bd4a854e65a9e8d5: Bootstrap starting.
I20260809 22:58:24.764746 15814 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 22:58:24.766179 18918 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cfe847d21d174998bd4a854e65a9e8d5: Neither blocks nor log segments found. Creating new log.
I20260809 22:58:24.767426 18918 log.cc:824] T 00000000000000000000000000000000 P cfe847d21d174998bd4a854e65a9e8d5: Log is configured to *not* fsync() on all Append() calls
I20260809 22:58:24.769970 18918 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cfe847d21d174998bd4a854e65a9e8d5: No bootstrap required, opened a new log
I20260809 22:58:24.767652 18651 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "8f81ddb676094a5b9e8f78b0d6e7ee3b"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46782
I20260809 22:58:24.776736 18918 raft_consensus.cc:348] T 00000000000000000000000000000000 P cfe847d21d174998bd4a854e65a9e8d5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f81ddb676094a5b9e8f78b0d6e7ee3b" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 42379 } } peers { permanent_uuid: "d51e830c048242209eb6513f255daa90" member_type: VOTER last_known_addr { host: "127.15.113.189" port: 38973 } } peers { permanent_uuid: "cfe847d21d174998bd4a854e65a9e8d5" member_type: VOTER last_known_addr { host: "127.15.113.188" port: 40915 } }
I20260809 22:58:24.777070 18918 raft_consensus.cc:729] T 00000000000000000000000000000000 P cfe847d21d174998bd4a854e65a9e8d5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cfe847d21d174998bd4a854e65a9e8d5, State: Initialized, Role: FOLLOWER
I20260809 22:58:24.777477 18918 consensus_queue.cc:260] T 00000000000000000000000000000000 P cfe847d21d174998bd4a854e65a9e8d5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f81ddb676094a5b9e8f78b0d6e7ee3b" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 42379 } } peers { permanent_uuid: "d51e830c048242209eb6513f255daa90" member_type: VOTER last_known_addr { host: "127.15.113.189" port: 38973 } } peers { permanent_uuid: "cfe847d21d174998bd4a854e65a9e8d5" member_type: VOTER last_known_addr { host: "127.15.113.188" port: 40915 } }
I20260809 22:58:24.778352 18933 sys_catalog.cc:455] T 00000000000000000000000000000000 P cfe847d21d174998bd4a854e65a9e8d5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f81ddb676094a5b9e8f78b0d6e7ee3b" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 42379 } } peers { permanent_uuid: "d51e830c048242209eb6513f255daa90" member_type: VOTER last_known_addr { host: "127.15.113.189" port: 38973 } } peers { permanent_uuid: "cfe847d21d174998bd4a854e65a9e8d5" member_type: VOTER last_known_addr { host: "127.15.113.188" port: 40915 } } }
I20260809 22:58:24.778733 18933 sys_catalog.cc:458] T 00000000000000000000000000000000 P cfe847d21d174998bd4a854e65a9e8d5 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 22:58:24.779251 18918 sys_catalog.cc:564] T 00000000000000000000000000000000 P cfe847d21d174998bd4a854e65a9e8d5 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 22:58:24.782174 18688 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.113.188" port: 40915 } attempt: 4
Aug 09 22:58:24 dist-test-slave-ncp9 krb5kdc[18574](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316303, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.113.190@KRBTEST.COM for kudu/127.15.113.188@KRBTEST.COM
W20260809 22:58:24.794960 18951 catalog_manager.cc:1558] T 00000000000000000000000000000000 P cfe847d21d174998bd4a854e65a9e8d5: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 22:58:24.795074 18951 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 22:58:24.797142 18690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:58:24.782833 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.188:40915 (local address 127.0.0.1:48910)
0809 22:58:24.783023 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0809 22:58:24.783044 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0809 22:58:24.783170 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:58:24.783495 (+ 325us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:58:24.783500 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:58:24.783899 (+ 399us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:58:24.784062 (+ 163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:58:24.784071 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:58:24.785093 (+ 1022us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:58:24.785099 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:58:24.786042 (+ 943us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:58:24.786054 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:58:24.786801 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:58:24.786808 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:58:24.786999 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:58:24.787600 (+ 601us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:58:24.787630 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:58:24.790337 (+ 2707us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:58:24.794232 (+ 3895us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:58:24.794238 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:58:24.794242 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:58:24.794734 (+ 492us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:58:24.795326 (+ 592us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:58:24.795332 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:58:24.795336 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:58:24.795504 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:58:24.796113 (+ 609us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:58:24.796118 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:58:24.796318 (+ 200us) client_negotiation.cc:769] Sending connection context
0809 22:58:24.796459 (+ 141us) client_negotiation.cc:240] Negotiation successful
0809 22:58:24.796730 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":48}
I20260809 22:58:24.797199 18923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:58:24.782875 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48910 (local address 127.15.113.188:40915)
0809 22:58:24.783042 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0809 22:58:24.783048 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:58:24.783098 (+ 50us) server_negotiation.cc:407] Connection header received
0809 22:58:24.783282 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:58:24.783288 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:58:24.783361 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:58:24.783447 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:58:24.784206 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:58:24.784944 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:58:24.786183 (+ 1239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:58:24.786653 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:58:24.786759 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:58:24.790498 (+ 3739us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:58:24.790504 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:58:24.790509 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:58:24.790543 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:58:24.794070 (+ 3527us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:58:24.794918 (+ 848us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:58:24.794924 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:58:24.794927 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:58:24.795099 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:58:24.795615 (+ 516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:58:24.795631 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:58:24.795635 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:58:24.795990 (+ 355us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:58:24.796162 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0809 22:58:24.796584 (+ 422us) server_negotiation.cc:299] Negotiation successful
0809 22:58:24.796814 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260809 22:58:24.801719 18688 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8f81ddb676094a5b9e8f78b0d6e7ee3b: Bootstrap starting.
I20260809 22:58:24.792853 18812 master_runner.cc:186] Error getting master registration for 127.15.113.190:42379: OK, instance_id { permanent_uuid: "8f81ddb676094a5b9e8f78b0d6e7ee3b" instance_seqno: 1786316303648361 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 22:58:24.803781 18812 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260809 22:58:24.805532 18688 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8f81ddb676094a5b9e8f78b0d6e7ee3b: Neither blocks nor log segments found. Creating new log.
I20260809 22:58:24.806787 18688 log.cc:824] T 00000000000000000000000000000000 P 8f81ddb676094a5b9e8f78b0d6e7ee3b: Log is configured to *not* fsync() on all Append() calls
I20260809 22:58:24.809967 18688 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8f81ddb676094a5b9e8f78b0d6e7ee3b: No bootstrap required, opened a new log
I20260809 22:58:24.817408 18688 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8f81ddb676094a5b9e8f78b0d6e7ee3b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f81ddb676094a5b9e8f78b0d6e7ee3b" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 42379 } } peers { permanent_uuid: "d51e830c048242209eb6513f255daa90" member_type: VOTER last_known_addr { host: "127.15.113.189" port: 38973 } } peers { permanent_uuid: "cfe847d21d174998bd4a854e65a9e8d5" member_type: VOTER last_known_addr { host: "127.15.113.188" port: 40915 } }
I20260809 22:58:24.817821 18688 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8f81ddb676094a5b9e8f78b0d6e7ee3b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f81ddb676094a5b9e8f78b0d6e7ee3b, State: Initialized, Role: FOLLOWER
I20260809 22:58:24.818316 18688 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8f81ddb676094a5b9e8f78b0d6e7ee3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f81ddb676094a5b9e8f78b0d6e7ee3b" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 42379 } } peers { permanent_uuid: "d51e830c048242209eb6513f255daa90" member_type: VOTER last_known_addr { host: "127.15.113.189" port: 38973 } } peers { permanent_uuid: "cfe847d21d174998bd4a854e65a9e8d5" member_type: VOTER last_known_addr { host: "127.15.113.188" port: 40915 } }
I20260809 22:58:24.819291 18956 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8f81ddb676094a5b9e8f78b0d6e7ee3b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f81ddb676094a5b9e8f78b0d6e7ee3b" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 42379 } } peers { permanent_uuid: "d51e830c048242209eb6513f255daa90" member_type: VOTER last_known_addr { host: "127.15.113.189" port: 38973 } } peers { permanent_uuid: "cfe847d21d174998bd4a854e65a9e8d5" member_type: VOTER last_known_addr { host: "127.15.113.188" port: 40915 } } }
I20260809 22:58:24.819739 18956 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8f81ddb676094a5b9e8f78b0d6e7ee3b [sys.catalog]: This master's current role is: FOLLOWER
I20260809 22:58:24.820480 18688 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8f81ddb676094a5b9e8f78b0d6e7ee3b [sys.catalog]: configured and running, proceeding with master startup.
I20260809 22:58:24.820727 18651 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8f81ddb676094a5b9e8f78b0d6e7ee3b [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260809 22:58:24.820979 18651 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8f81ddb676094a5b9e8f78b0d6e7ee3b [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:58:24.824690 18651 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8f81ddb676094a5b9e8f78b0d6e7ee3b [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f81ddb676094a5b9e8f78b0d6e7ee3b" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 42379 } } peers { permanent_uuid: "d51e830c048242209eb6513f255daa90" member_type: VOTER last_known_addr { host: "127.15.113.189" port: 38973 } } peers { permanent_uuid: "cfe847d21d174998bd4a854e65a9e8d5" member_type: VOTER last_known_addr { host: "127.15.113.188" port: 40915 } }
I20260809 22:58:24.828703 18878 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "8f81ddb676094a5b9e8f78b0d6e7ee3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "cfe847d21d174998bd4a854e65a9e8d5"
I20260809 22:58:24.829250 18878 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cfe847d21d174998bd4a854e65a9e8d5 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:58:24.833565 18878 raft_consensus.cc:2454] T 00000000000000000000000000000000 P cfe847d21d174998bd4a854e65a9e8d5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f81ddb676094a5b9e8f78b0d6e7ee3b in term 1.
I20260809 22:58:24.834398 18619 leader_election.cc:302] T 00000000000000000000000000000000 P 8f81ddb676094a5b9e8f78b0d6e7ee3b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8f81ddb676094a5b9e8f78b0d6e7ee3b, cfe847d21d174998bd4a854e65a9e8d5; no voters:
I20260809 22:58:24.834888 18956 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8f81ddb676094a5b9e8f78b0d6e7ee3b [term 1 FOLLOWER]: Leader election won for term 1
I20260809 22:58:24.830220 18592 master_runner.cc:186] Error getting master registration for 127.15.113.189:38973: OK, instance_id { permanent_uuid: "d51e830c048242209eb6513f255daa90" instance_seqno: 1786316304164349 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 22:58:24.840870 18592 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260809 22:58:24.841816 18651 leader_election.cc:288] T 00000000000000000000000000000000 P 8f81ddb676094a5b9e8f78b0d6e7ee3b [CANDIDATE]: Term 1 election: Requested vote from peers d51e830c048242209eb6513f255daa90 (127.15.113.189:38973), cfe847d21d174998bd4a854e65a9e8d5 (127.15.113.188:40915)
I20260809 22:58:24.845418 18956 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8f81ddb676094a5b9e8f78b0d6e7ee3b [term 1 LEADER]: Becoming Leader. State: Replica: 8f81ddb676094a5b9e8f78b0d6e7ee3b, State: Running, Role: LEADER
I20260809 22:58:24.846077 18956 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8f81ddb676094a5b9e8f78b0d6e7ee3b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f81ddb676094a5b9e8f78b0d6e7ee3b" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 42379 } } peers { permanent_uuid: "d51e830c048242209eb6513f255daa90" member_type: VOTER last_known_addr { host: "127.15.113.189" port: 38973 } } peers { permanent_uuid: "cfe847d21d174998bd4a854e65a9e8d5" member_type: VOTER last_known_addr { host: "127.15.113.188" port: 40915 } }
I20260809 22:58:24.856020 18975 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8f81ddb676094a5b9e8f78b0d6e7ee3b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8f81ddb676094a5b9e8f78b0d6e7ee3b. Latest consensus state: current_term: 1 leader_uuid: "8f81ddb676094a5b9e8f78b0d6e7ee3b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f81ddb676094a5b9e8f78b0d6e7ee3b" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 42379 } } peers { permanent_uuid: "d51e830c048242209eb6513f255daa90" member_type: VOTER last_known_addr { host: "127.15.113.189" port: 38973 } } peers { permanent_uuid: "cfe847d21d174998bd4a854e65a9e8d5" member_type: VOTER last_known_addr { host: "127.15.113.188" port: 40915 } } }
I20260809 22:58:24.856508 18975 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8f81ddb676094a5b9e8f78b0d6e7ee3b [sys.catalog]: This master's current role is: LEADER
W20260809 22:58:24.857653 18971 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 8f81ddb676094a5b9e8f78b0d6e7ee3b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 22:58:24.857765 18971 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 22:58:24.873448 18984 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 22:58:24.875502 18984 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 22:58:24.887243 18805 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.113.188" port: 40915 } attempt: 5
W20260809 22:58:24.888784 18623 tablet.cc:2369] T 00000000000000000000000000000000 P 8f81ddb676094a5b9e8f78b0d6e7ee3b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 22:58:24.909780 18878 raft_consensus.cc:1261] T 00000000000000000000000000000000 P cfe847d21d174998bd4a854e65a9e8d5 [term 1 FOLLOWER]: Refusing update from remote peer 8f81ddb676094a5b9e8f78b0d6e7ee3b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 22:58:24.911077 18975 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 8f81ddb676094a5b9e8f78b0d6e7ee3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "cfe847d21d174998bd4a854e65a9e8d5" member_type: VOTER last_known_addr { host: "127.15.113.188" port: 40915 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 22:58:24.912422 18928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:58:24.883833 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46786 (local address 127.15.113.190:42379)
0809 22:58:24.884117 (+ 284us) server_negotiation.cc:206] Beginning negotiation
0809 22:58:24.884129 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 22:58:24.885111 (+ 982us) server_negotiation.cc:407] Connection header received
0809 22:58:24.885547 (+ 436us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:58:24.885553 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:58:24.885638 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:58:24.885713 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:58:24.889768 (+ 4055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:58:24.890891 (+ 1123us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:58:24.894050 (+ 3159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:58:24.894640 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:58:24.894732 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:58:24.900509 (+ 5777us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:58:24.900516 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:58:24.900520 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:58:24.900559 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:58:24.904163 (+ 3604us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:58:24.907965 (+ 3802us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:58:24.907972 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:58:24.907976 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:58:24.908129 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:58:24.909478 (+ 1349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:58:24.909483 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:58:24.909486 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:58:24.909857 (+ 371us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:58:24.910133 (+ 276us) server_negotiation.cc:1035] Waiting for connection context
0809 22:58:24.911800 (+ 1667us) server_negotiation.cc:299] Negotiation successful
0809 22:58:24.912059 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":128}
I20260809 22:58:24.913208 18809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:58:24.883107 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44758 (local address 127.15.113.189:38973)
0809 22:58:24.883306 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0809 22:58:24.883313 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:58:24.890126 (+ 6813us) server_negotiation.cc:407] Connection header received
0809 22:58:24.890563 (+ 437us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:58:24.890569 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:58:24.890647 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:58:24.890710 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:58:24.893070 (+ 2360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:58:24.894137 (+ 1067us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:58:24.896586 (+ 2449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:58:24.897227 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:58:24.897326 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:58:24.904265 (+ 6939us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:58:24.904270 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:58:24.904274 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:58:24.904312 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:58:24.907792 (+ 3480us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:58:24.910742 (+ 2950us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:58:24.910748 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:58:24.910751 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:58:24.910897 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:58:24.911404 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:58:24.911410 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:58:24.911413 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:58:24.911775 (+ 362us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:58:24.912005 (+ 230us) server_negotiation.cc:1035] Waiting for connection context
0809 22:58:24.912605 (+ 600us) server_negotiation.cc:299] Negotiation successful
0809 22:58:24.912830 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
Aug 09 22:58:24 dist-test-slave-ncp9 krb5kdc[18574](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316304, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.113.189@KRBTEST.COM for kudu/127.15.113.188@KRBTEST.COM
I20260809 22:58:24.928931 18807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:58:24.887797 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.113.188:40915 (local address 127.0.0.1:48926)
0809 22:58:24.888040 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0809 22:58:24.888059 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 22:58:24.888160 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:58:24.906975 (+ 18815us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:58:24.906981 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:58:24.908082 (+ 1101us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:58:24.908279 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:58:24.908291 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:58:24.913675 (+ 5384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:58:24.913681 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:58:24.915083 (+ 1402us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:58:24.915098 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:58:24.918085 (+ 2987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:58:24.918091 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:58:24.918326 (+ 235us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:58:24.918890 (+ 564us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:58:24.918911 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:58:24.921628 (+ 2717us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:58:24.926129 (+ 4501us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:58:24.926135 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:58:24.926139 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:58:24.926637 (+ 498us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:58:24.927106 (+ 469us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:58:24.927111 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:58:24.927115 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:58:24.927279 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:58:24.927938 (+ 659us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:58:24.927943 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:58:24.928086 (+ 143us) client_negotiation.cc:769] Sending connection context
0809 22:58:24.928227 (+ 141us) client_negotiation.cc:240] Negotiation successful
0809 22:58:24.928523 (+ 296us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137}
W20260809 22:58:24.929908 18842 tablet.cc:2369] T 00000000000000000000000000000000 P cfe847d21d174998bd4a854e65a9e8d5: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 22:58:24.930383 18996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:58:24.888452 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48926 (local address 127.15.113.188:40915)
0809 22:58:24.906634 (+ 18182us) server_negotiation.cc:206] Beginning negotiation
0809 22:58:24.906639 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:58:24.906662 (+ 23us) server_negotiation.cc:407] Connection header received
0809 22:58:24.906708 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:58:24.906713 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:58:24.906777 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:58:24.907419 (+ 642us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:58:24.909025 (+ 1606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:58:24.910335 (+ 1310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:58:24.917184 (+ 6849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:58:24.917914 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:58:24.918615 (+ 701us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:58:24.921874 (+ 3259us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:58:24.921881 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:58:24.921886 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:58:24.921924 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:58:24.925980 (+ 4056us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:58:24.926789 (+ 809us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:58:24.926794 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:58:24.926797 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:58:24.926959 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:58:24.927420 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:58:24.927425 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:58:24.927429 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:58:24.927792 (+ 363us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:58:24.929287 (+ 1495us) server_negotiation.cc:1035] Waiting for connection context
0809 22:58:24.929442 (+ 155us) server_negotiation.cc:299] Negotiation successful
0809 22:58:24.929964 (+ 522us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17975,"thread_start_us":17550,"threads_started":1}
I20260809 22:58:24.934333 18933 sys_catalog.cc:455] T 00000000000000000000000000000000 P cfe847d21d174998bd4a854e65a9e8d5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8f81ddb676094a5b9e8f78b0d6e7ee3b. Latest consensus state: current_term: 1 leader_uuid: "8f81ddb676094a5b9e8f78b0d6e7ee3b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f81ddb676094a5b9e8f78b0d6e7ee3b" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 42379 } } peers { permanent_uuid: "d51e830c048242209eb6513f255daa90" member_type: VOTER last_known_addr { host: "127.15.113.189" port: 38973 } } peers { permanent_uuid: "cfe847d21d174998bd4a854e65a9e8d5" member_type: VOTER last_known_addr { host: "127.15.113.188" port: 40915 } } }
I20260809 22:58:24.934700 18933 sys_catalog.cc:458] T 00000000000000000000000000000000 P cfe847d21d174998bd4a854e65a9e8d5 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 22:58:24.935367 18923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:58:24.883869 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48920 (local address 127.15.113.188:40915)
0809 22:58:24.884272 (+ 403us) server_negotiation.cc:206] Beginning negotiation
0809 22:58:24.884278 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:58:24.904816 (+ 20538us) server_negotiation.cc:407] Connection header received
0809 22:58:24.905780 (+ 964us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:58:24.905787 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:58:24.905904 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:58:24.906231 (+ 327us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:58:24.914354 (+ 8123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:58:24.915367 (+ 1013us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:58:24.918284 (+ 2917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:58:24.919275 (+ 991us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:58:24.919371 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:58:24.925845 (+ 6474us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:58:24.925852 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:58:24.925856 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:58:24.925904 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:58:24.929795 (+ 3891us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:58:24.932011 (+ 2216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:58:24.932017 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:58:24.932020 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:58:24.932174 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:58:24.932989 (+ 815us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:58:24.932995 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:58:24.932998 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:58:24.933330 (+ 332us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:58:24.933630 (+ 300us) server_negotiation.cc:1035] Waiting for connection context
0809 22:58:24.934668 (+ 1038us) server_negotiation.cc:299] Negotiation successful
0809 22:58:24.934902 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":279}
I20260809 22:58:24.936223 18805 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d51e830c048242209eb6513f255daa90: Bootstrap starting.
I20260809 22:58:24.937145 18994 mvcc.cc:205] Tried to move back new op lower bound from 7316751584875679744 to 7316751584655519744. Current Snapshot: MvccSnapshot[applied={T|T < 7316751584875679744}]
I20260809 22:58:24.938949 18975 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8f81ddb676094a5b9e8f78b0d6e7ee3b [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "8f81ddb676094a5b9e8f78b0d6e7ee3b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f81ddb676094a5b9e8f78b0d6e7ee3b" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 42379 } } peers { permanent_uuid: "d51e830c048242209eb6513f255daa90" member_type: VOTER last_known_addr { host: "127.15.113.189" port: 38973 } } peers { permanent_uuid: "cfe847d21d174998bd4a854e65a9e8d5" member_type: VOTER last_known_addr { host: "127.15.113.188" port: 40915 } } }
I20260809 22:58:24.939226 18975 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8f81ddb676094a5b9e8f78b0d6e7ee3b [sys.catalog]: This master's current role is: LEADER
I20260809 22:58:24.941206 18997 mvcc.cc:205] Tried to move back new op lower bound from 7316751584875679744 to 7316751584655519744. Current Snapshot: MvccSnapshot[applied={T|T < 7316751584875679744}]
I20260809 22:58:24.942087 18984 catalog_manager.cc:1347] Generated new cluster ID: bd78a2e7dcec49f58467c660d344ac9c
I20260809 22:58:24.942200 18984 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 22:58:24.942252 18805 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d51e830c048242209eb6513f255daa90: Neither blocks nor log segments found. Creating new log.
I20260809 22:58:24.946558 18805 log.cc:824] T 00000000000000000000000000000000 P d51e830c048242209eb6513f255daa90: Log is configured to *not* fsync() on all Append() calls
I20260809 22:58:24.950306 18933 sys_catalog.cc:455] T 00000000000000000000000000000000 P cfe847d21d174998bd4a854e65a9e8d5 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "8f81ddb676094a5b9e8f78b0d6e7ee3b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f81ddb676094a5b9e8f78b0d6e7ee3b" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 42379 } } peers { permanent_uuid: "d51e830c048242209eb6513f255daa90" member_type: VOTER last_known_addr { host: "127.15.113.189" port: 38973 } } peers { permanent_uuid: "cfe847d21d174998bd4a854e65a9e8d5" member_type: VOTER last_known_addr { host: "127.15.113.188" port: 40915 } } }
I20260809 22:58:24.950542 18933 sys_catalog.cc:458] T 00000000000000000000000000000000 P cfe847d21d174998bd4a854e65a9e8d5 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 22:58:24.963537 18805 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d51e830c048242209eb6513f255daa90: No bootstrap required, opened a new log
I20260809 22:58:24.973307 18984 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 22:58:24.975106 18984 catalog_manager.cc:1504] Loading token signing keys...
I20260809 22:58:24.986763 18805 raft_consensus.cc:348] T 00000000000000000000000000000000 P d51e830c048242209eb6513f255daa90 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f81ddb676094a5b9e8f78b0d6e7ee3b" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 42379 } } peers { permanent_uuid: "d51e830c048242209eb6513f255daa90" member_type: VOTER last_known_addr { host: "127.15.113.189" port: 38973 } } peers { permanent_uuid: "cfe847d21d174998bd4a854e65a9e8d5" member_type: VOTER last_known_addr { host: "127.15.113.188" port: 40915 } }
I20260809 22:58:24.987253 18805 raft_consensus.cc:729] T 00000000000000000000000000000000 P d51e830c048242209eb6513f255daa90 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d51e830c048242209eb6513f255daa90, State: Initialized, Role: FOLLOWER
I20260809 22:58:24.987830 18805 consensus_queue.cc:260] T 00000000000000000000000000000000 P d51e830c048242209eb6513f255daa90 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f81ddb676094a5b9e8f78b0d6e7ee3b" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 42379 } } peers { permanent_uuid: "d51e830c048242209eb6513f255daa90" member_type: VOTER last_known_addr { host: "127.15.113.189" port: 38973 } } peers { permanent_uuid: "cfe847d21d174998bd4a854e65a9e8d5" member_type: VOTER last_known_addr { host: "127.15.113.188" port: 40915 } }
I20260809 22:58:24.989105 18984 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8f81ddb676094a5b9e8f78b0d6e7ee3b: Generated new TSK 0
I20260809 22:58:24.990183 18984 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 22:58:24.991940 19005 sys_catalog.cc:455] T 00000000000000000000000000000000 P d51e830c048242209eb6513f255daa90 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f81ddb676094a5b9e8f78b0d6e7ee3b" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 42379 } } peers { permanent_uuid: "d51e830c048242209eb6513f255daa90" member_type: VOTER last_known_addr { host: "127.15.113.189" port: 38973 } } peers { permanent_uuid: "cfe847d21d174998bd4a854e65a9e8d5" member_type: VOTER last_known_addr { host: "127.15.113.188" port: 40915 } } }
I20260809 22:58:24.992363 19005 sys_catalog.cc:458] T 00000000000000000000000000000000 P d51e830c048242209eb6513f255daa90 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 22:58:24.994163 18805 sys_catalog.cc:564] T 00000000000000000000000000000000 P d51e830c048242209eb6513f255daa90 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 22:58:24.994261 18768 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "8f81ddb676094a5b9e8f78b0d6e7ee3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "d51e830c048242209eb6513f255daa90"
I20260809 22:58:24.994807 18768 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d51e830c048242209eb6513f255daa90 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:58:25.007804 18768 raft_consensus.cc:2454] T 00000000000000000000000000000000 P d51e830c048242209eb6513f255daa90 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f81ddb676094a5b9e8f78b0d6e7ee3b in term 1.
I20260809 22:58:25.011740 18767 raft_consensus.cc:1261] T 00000000000000000000000000000000 P d51e830c048242209eb6513f255daa90 [term 1 FOLLOWER]: Refusing update from remote peer 8f81ddb676094a5b9e8f78b0d6e7ee3b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 22:58:25.013584 18975 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 8f81ddb676094a5b9e8f78b0d6e7ee3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51e830c048242209eb6513f255daa90" member_type: VOTER last_known_addr { host: "127.15.113.189" port: 38973 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 22:58:25.063611 19005 sys_catalog.cc:455] T 00000000000000000000000000000000 P d51e830c048242209eb6513f255daa90 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8f81ddb676094a5b9e8f78b0d6e7ee3b. Latest consensus state: current_term: 1 leader_uuid: "8f81ddb676094a5b9e8f78b0d6e7ee3b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f81ddb676094a5b9e8f78b0d6e7ee3b" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 42379 } } peers { permanent_uuid: "d51e830c048242209eb6513f255daa90" member_type: VOTER last_known_addr { host: "127.15.113.189" port: 38973 } } peers { permanent_uuid: "cfe847d21d174998bd4a854e65a9e8d5" member_type: VOTER last_known_addr { host: "127.15.113.188" port: 40915 } } }
I20260809 22:58:25.063911 19005 sys_catalog.cc:458] T 00000000000000000000000000000000 P d51e830c048242209eb6513f255daa90 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 22:58:25.064417 19019 sys_catalog.cc:455] T 00000000000000000000000000000000 P d51e830c048242209eb6513f255daa90 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "8f81ddb676094a5b9e8f78b0d6e7ee3b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f81ddb676094a5b9e8f78b0d6e7ee3b" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 42379 } } peers { permanent_uuid: "d51e830c048242209eb6513f255daa90" member_type: VOTER last_known_addr { host: "127.15.113.189" port: 38973 } } peers { permanent_uuid: "cfe847d21d174998bd4a854e65a9e8d5" member_type: VOTER last_known_addr { host: "127.15.113.188" port: 40915 } } }
I20260809 22:58:25.064810 19019 sys_catalog.cc:458] T 00000000000000000000000000000000 P d51e830c048242209eb6513f255daa90 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 22:58:25.085780 19030 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
W20260809 22:58:25.115468 18809 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:58:25.099557 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44772 (local address 127.15.113.189:38973)
0809 22:58:25.099907 (+ 350us) server_negotiation.cc:206] Beginning negotiation
0809 22:58:25.099913 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:58:25.113508 (+ 13595us) server_negotiation.cc:407] Connection header received
0809 22:58:25.113800 (+ 292us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:58:25.113806 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:58:25.113901 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:58:25.113972 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:58:25.115226 (+ 1254us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44772: BlockingRecv error: recv got EOF from 127.0.0.1:44772 (error 108)
Metrics: {"server-negotiator.queue_time_us":236}
W20260809 22:58:25.117890 18928 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:58:25.099373 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46800 (local address 127.15.113.190:42379)
0809 22:58:25.099767 (+ 394us) server_negotiation.cc:206] Beginning negotiation
0809 22:58:25.099773 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:58:25.116116 (+ 16343us) server_negotiation.cc:407] Connection header received
0809 22:58:25.116351 (+ 235us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:58:25.116357 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:58:25.116439 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:58:25.116508 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:58:25.117675 (+ 1167us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46800: BlockingRecv error: recv got EOF from 127.0.0.1:46800 (error 108)
Metrics: {"server-negotiator.queue_time_us":246}
W20260809 22:58:25.119632 18923 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:58:25.101143 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48942 (local address 127.15.113.188:40915)
0809 22:58:25.101322 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0809 22:58:25.101328 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:58:25.117019 (+ 15691us) server_negotiation.cc:407] Connection header received
0809 22:58:25.118088 (+ 1069us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:58:25.118093 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:58:25.118180 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:58:25.118260 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:58:25.119327 (+ 1067us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48942: BlockingRecv error: recv got EOF from 127.0.0.1:48942 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
W20260809 22:58:25.125468 18809 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:58:25.123327 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44778 (local address 127.15.113.189:38973)
0809 22:58:25.123530 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0809 22:58:25.123536 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:58:25.124017 (+ 481us) server_negotiation.cc:407] Connection header received
0809 22:58:25.124327 (+ 310us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:58:25.124331 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:58:25.124425 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:58:25.124522 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:58:25.125245 (+ 723us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44778: BlockingRecv error: recv got EOF from 127.0.0.1:44778 (error 108)
Metrics: {"server-negotiator.queue_time_us":84}
W20260809 22:58:25.132507 18809 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:58:25.130785 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44792 (local address 127.15.113.189:38973)
0809 22:58:25.130950 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0809 22:58:25.130956 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:58:25.131377 (+ 421us) server_negotiation.cc:407] Connection header received
0809 22:58:25.131544 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:58:25.131549 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:58:25.131616 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:58:25.131691 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:58:25.132327 (+ 636us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44792: BlockingRecv error: recv got EOF from 127.0.0.1:44792 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
W20260809 22:58:25.139043 18809 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:58:25.137229 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44796 (local address 127.15.113.189:38973)
0809 22:58:25.137394 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0809 22:58:25.137400 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:58:25.137845 (+ 445us) server_negotiation.cc:407] Connection header received
0809 22:58:25.138017 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:58:25.138022 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:58:25.138095 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:58:25.138192 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:58:25.138828 (+ 636us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44796: BlockingRecv error: recv got EOF from 127.0.0.1:44796 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
W20260809 22:58:25.147620 18809 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:58:25.145811 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44798 (local address 127.15.113.189:38973)
0809 22:58:25.146036 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0809 22:58:25.146042 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:58:25.146483 (+ 441us) server_negotiation.cc:407] Connection header received
0809 22:58:25.146625 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:58:25.146630 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:58:25.146698 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:58:25.146776 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:58:25.147405 (+ 629us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44798: BlockingRecv error: recv got EOF from 127.0.0.1:44798 (error 108)
Metrics: {"server-negotiator.queue_time_us":97}
W20260809 22:58:25.160240 18809 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:58:25.158480 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44802 (local address 127.15.113.189:38973)
0809 22:58:25.158639 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0809 22:58:25.158645 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:58:25.159112 (+ 467us) server_negotiation.cc:407] Connection header received
0809 22:58:25.159286 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:58:25.159292 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:58:25.159354 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:58:25.159432 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:58:25.160042 (+ 610us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44802: BlockingRecv error: recv got EOF from 127.0.0.1:44802 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
W20260809 22:58:25.196038 18809 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:58:25.193860 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44816 (local address 127.15.113.189:38973)
0809 22:58:25.194099 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0809 22:58:25.194105 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:58:25.194644 (+ 539us) server_negotiation.cc:407] Connection header received
0809 22:58:25.194853 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:58:25.194857 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:58:25.194938 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:58:25.195026 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:58:25.195836 (+ 810us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44816: BlockingRecv error: recv got EOF from 127.0.0.1:44816 (error 108)
Metrics: {"server-negotiator.queue_time_us":88}
W20260809 22:58:25.233711 18809 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:58:25.231490 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44822 (local address 127.15.113.189:38973)
0809 22:58:25.231701 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0809 22:58:25.231709 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 22:58:25.232222 (+ 513us) server_negotiation.cc:407] Connection header received
0809 22:58:25.232477 (+ 255us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:58:25.232481 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:58:25.232552 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:58:25.232636 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:58:25.233433 (+ 797us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44822: BlockingRecv error: recv got EOF from 127.0.0.1:44822 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
W20260809 22:58:25.303186 18809 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:58:25.301033 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44830 (local address 127.15.113.189:38973)
0809 22:58:25.301302 (+ 269us) server_negotiation.cc:206] Beginning negotiation
0809 22:58:25.301309 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:58:25.301819 (+ 510us) server_negotiation.cc:407] Connection header received
0809 22:58:25.302048 (+ 229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:58:25.302055 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:58:25.302134 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:58:25.302232 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:58:25.302977 (+ 745us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44830: BlockingRecv error: recv got EOF from 127.0.0.1:44830 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
W20260809 22:58:25.436708 18809 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:58:25.434709 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44832 (local address 127.15.113.189:38973)
0809 22:58:25.434888 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0809 22:58:25.434894 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:58:25.435264 (+ 370us) server_negotiation.cc:407] Connection header received
0809 22:58:25.435464 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:58:25.435468 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:58:25.435549 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:58:25.435636 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:58:25.436432 (+ 796us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44832: BlockingRecv error: recv got EOF from 127.0.0.1:44832 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
I20260809 22:58:25.529956 18975 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8f81ddb676094a5b9e8f78b0d6e7ee3b [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "8f81ddb676094a5b9e8f78b0d6e7ee3b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f81ddb676094a5b9e8f78b0d6e7ee3b" member_type: VOTER last_known_addr { host: "127.15.113.190" port: 42379 } } peers { permanent_uuid: "d51e830c048242209eb6513f255daa90" member_type: VOTER last_known_addr { host: "127.15.113.189" port: 38973 } } peers { permanent_uuid: "cfe847d21d174998bd4a854e65a9e8d5" member_type: VOTER last_known_addr { host: "127.15.113.188" port: 40915 } } }
I20260809 22:58:25.530294 18975 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8f81ddb676094a5b9e8f78b0d6e7ee3b [sys.catalog]: This master's current role is: LEADER
W20260809 22:58:25.698439 18809 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:58:25.696560 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44846 (local address 127.15.113.189:38973)
0809 22:58:25.696753 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0809 22:58:25.696759 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:58:25.697067 (+ 308us) server_negotiation.cc:407] Connection header received
0809 22:58:25.697298 (+ 231us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:58:25.697302 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:58:25.697375 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:58:25.697461 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:58:25.698228 (+ 767us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44846: BlockingRecv error: recv got EOF from 127.0.0.1:44846 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
I20260809 22:58:25.797092 18951 catalog_manager.cc:1259] Loaded cluster ID: bd78a2e7dcec49f58467c660d344ac9c
I20260809 22:58:25.797200 18951 catalog_manager.cc:1552] T 00000000000000000000000000000000 P cfe847d21d174998bd4a854e65a9e8d5: loading cluster ID for follower catalog manager: success
I20260809 22:58:25.799628 18951 catalog_manager.cc:1574] T 00000000000000000000000000000000 P cfe847d21d174998bd4a854e65a9e8d5: acquiring CA information for follower catalog manager: success
I20260809 22:58:25.801429 18951 catalog_manager.cc:1602] T 00000000000000000000000000000000 P cfe847d21d174998bd4a854e65a9e8d5: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 22:58:26.090265 19030 catalog_manager.cc:1259] Loaded cluster ID: bd78a2e7dcec49f58467c660d344ac9c
I20260809 22:58:26.090382 19030 catalog_manager.cc:1552] T 00000000000000000000000000000000 P d51e830c048242209eb6513f255daa90: loading cluster ID for follower catalog manager: success
I20260809 22:58:26.092981 19030 catalog_manager.cc:1574] T 00000000000000000000000000000000 P d51e830c048242209eb6513f255daa90: acquiring CA information for follower catalog manager: success
I20260809 22:58:26.094776 19030 catalog_manager.cc:1602] T 00000000000000000000000000000000 P d51e830c048242209eb6513f255daa90: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 22:58:26.226179 19138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:58:26.218540 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44856 (local address 127.15.113.189:38973)
0809 22:58:26.219490 (+ 950us) server_negotiation.cc:206] Beginning negotiation
0809 22:58:26.219496 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:58:26.219835 (+ 339us) server_negotiation.cc:407] Connection header received
0809 22:58:26.220059 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:58:26.220064 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:58:26.220247 (+ 183us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:58:26.220357 (+ 110us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 22:58:26.220865 (+ 508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:58:26.221855 (+ 990us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:58:26.223827 (+ 1972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:58:26.224334 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:58:26.224430 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:58:26.225034 (+ 604us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 22:58:26.225567 (+ 533us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 22:58:26.225724 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0809 22:58:26.225811 (+ 87us) server_negotiation.cc:299] Negotiation successful
0809 22:58:26.225875 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":786,"thread_start_us":595,"threads_started":1}
I20260809 22:58:26.239152 19147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:58:26.231824 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48944 (local address 127.15.113.188:40915)
0809 22:58:26.233256 (+ 1432us) server_negotiation.cc:206] Beginning negotiation
0809 22:58:26.233261 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:58:26.233280 (+ 19us) server_negotiation.cc:407] Connection header received
0809 22:58:26.233388 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:58:26.233393 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:58:26.233597 (+ 204us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:58:26.233781 (+ 184us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 22:58:26.234127 (+ 346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:58:26.235230 (+ 1103us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:58:26.236430 (+ 1200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:58:26.237095 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:58:26.237222 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:58:26.237713 (+ 491us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 22:58:26.238531 (+ 818us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 22:58:26.238692 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0809 22:58:26.238746 (+ 54us) server_negotiation.cc:299] Negotiation successful
0809 22:58:26.238833 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1308,"thread_start_us":1189,"threads_started":1}
I20260809 22:58:26.258059 19147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:58:26.245304 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48950 (local address 127.15.113.188:40915)
0809 22:58:26.245462 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0809 22:58:26.245467 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:58:26.251950 (+ 6483us) server_negotiation.cc:407] Connection header received
0809 22:58:26.252127 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:58:26.252132 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:58:26.252229 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:58:26.252331 (+ 102us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 22:58:26.252788 (+ 457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:58:26.253832 (+ 1044us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:58:26.255888 (+ 2056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:58:26.256495 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:58:26.256586 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:58:26.257288 (+ 702us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 22:58:26.257423 (+ 135us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 22:58:26.257490 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0809 22:58:26.257709 (+ 219us) server_negotiation.cc:299] Negotiation successful
0809 22:58:26.257784 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260809 22:58:26.260426 19138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:58:26.244846 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44862 (local address 127.15.113.189:38973)
0809 22:58:26.245030 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0809 22:58:26.245035 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:58:26.253285 (+ 8250us) server_negotiation.cc:407] Connection header received
0809 22:58:26.253542 (+ 257us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:58:26.253548 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:58:26.253633 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:58:26.253697 (+ 64us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 22:58:26.254902 (+ 1205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:58:26.256139 (+ 1237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:58:26.258354 (+ 2215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:58:26.258946 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:58:26.259037 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:58:26.259686 (+ 649us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 22:58:26.259811 (+ 125us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 22:58:26.259874 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0809 22:58:26.260105 (+ 231us) server_negotiation.cc:299] Negotiation successful
0809 22:58:26.260185 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260809 22:58:26.263844 19155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:58:26.244859 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46808 (local address 127.15.113.190:42379)
0809 22:58:26.254375 (+ 9516us) server_negotiation.cc:206] Beginning negotiation
0809 22:58:26.254379 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 22:58:26.257649 (+ 3270us) server_negotiation.cc:407] Connection header received
0809 22:58:26.258103 (+ 454us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:58:26.258108 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:58:26.258336 (+ 228us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:58:26.258482 (+ 146us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 22:58:26.259428 (+ 946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:58:26.260457 (+ 1029us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:58:26.261986 (+ 1529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:58:26.262386 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:58:26.262498 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:58:26.262956 (+ 458us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 22:58:26.263281 (+ 325us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 22:58:26.263386 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0809 22:58:26.263513 (+ 127us) server_negotiation.cc:299] Negotiation successful
0809 22:58:26.263598 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9417,"thread_start_us":9290,"threads_started":1}
I20260809 22:58:26.268457 15814 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskItYPJR/build/asan/bin/kudu pid 18592
I20260809 22:58:26.269515 18681 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:58:26.377411 15814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskItYPJR/build/asan/bin/kudu with pid 18592
I20260809 22:58:26.408396 15814 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskItYPJR/build/asan/bin/kudu pid 18699
I20260809 22:58:26.409583 18799 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:58:26.523720 15814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskItYPJR/build/asan/bin/kudu with pid 18699
I20260809 22:58:26.547712 15814 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskItYPJR/build/asan/bin/kudu pid 18812
I20260809 22:58:26.548753 18910 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:58:26.655654 15814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskItYPJR/build/asan/bin/kudu with pid 18812
2026-08-09T22:58:26Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (4980 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (10032 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (371000 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST